Open to contributors.

Where to start

Clone the repo, sync the environment, install the hooks, and run the checks. Everything is local.

set up and run the checks

git clone https://github.com/Syntelesis-Lab/evalglass
uv sync
pre-commit install
uv run pytest
uv run ruff check .
uv run mypy

CLAUDE.md is the operating guide for the project — read it before you change anything load-bearing. It is written for humans and coding agents alike, and it spells out the boundaries below in full.

The rules that keep it honest

The brand is no false confidence, and a handful of boundaries protect it. A change that breaks one of these will not merge, however useful it looks.

• The Evaluation Core stays effect-free. Standard-library only: no file or network I/O, no clocks, no randomness, no SDKs. Effects live in the Runtime Harness.
• Only the Verdict Engine decides a verdict. No report, CLI, adapter, or sink may invent its own pass, fail, or blocked. Claim boundary →
• No silent gating. Scaffolded data, proposed thresholds, and uncalibrated judges start informational; they never quietly become authority.
• No domain knowledge in the core. The spine is generic by contract. Your domain judgment lives in your own host repo, not in the framework.
• Every dependency is a vendored cost. The core takes none; the runtime takes few and pinned; optional integrations stay isolated and deletable.
• One slice = one PR. Work ships in small, tested, reviewable slices rather than large drops.
• Hard-to-reverse decisions need an ADR. New dependencies, public-contract changes, and verdict or layout changes are recorded before they land. Governance →

Good first contributions

• File an issue. A reproducible bug, an unclear boundary, or a missing diagnostic is a real contribution. Open an issue →
• Propose an ADR. If you think a hard-to-reverse decision should be made differently, write it up so the reasoning is on the record.
• Add a metric in your own host repo. Host-owned evaluators are the intended extension point — build one, then tell us what the framework made hard.
• Improve the docs. Clearer prose, a fixed example, or a sharper diagram lowers the barrier for the next person.

Where discussion happens

Conversation lives on GitHub — Issues for concrete work, Discussions for open questions. There is no server-backed form and no separate channel to join. If something doesn't fit either, open an issue and we'll route it.