Releases
Changelog
EvalGlass Core's tagged release history, newest first — each entry summarized from its GitHub release, with GitHub Releases as the canonical source. EvalGlass is pre-alpha: versions are 0.x and contracts are still settling.
v0.2.1 · 9 Aug 2026
Trace-pull truncation warning & honest call counts
A patch release: two post-0.2.0 fixes plus launch housekeeping. No change to the verdict, authority, or schema contracts.
- Added — a truncation / representativeness warning. The opt-in Langfuse, Phoenix and LangSmith connectors list newest-first under a
limit, so a bounded pull can cover only the most-recent slice of a source. The shared connector boundary now emits<provider>_pull_truncatedwhen a known source total exceeds the pulled count, and a softer_pull_possibly_truncatedwhen a pull exactly fills its limit with no reported total. It is coverage evidence only — it never rejects a record and changes no score, authority, verdict, or CI exit. - Fixed — the discovery “N calls contributed” count. The headline (and the HTML KPI and
totals.subjects) was derived from the display-capped evidence sample, so a run over 200 contributing calls could read “9 calls contributed.” It now reads the true distinct-subject count. The error was in the safe direction — an under-count, never an overclaim — but an inaccurate headline is exactly the quiet unreliability the framework refuses. - Changed — public-launch housekeeping. A CodeQL analysis workflow and a Code of Conduct were added, and pre-launch documentation was corrected.
v0.2.0 · 9 Aug 2026
Advanced Discovery Metrics Engine
EvalGlass could already measure metrics a host had written; it could not tell a host which metrics to write. v0.2.0 adds an engine that reads recorded application behavior, derives a catalogue of metrics with stable identity, executes the ones it can, and reports what it measured, what it could not, and exactly what evidence would unlock the rest.
- It cannot gate, by construction.
DiscoveredMetricV2.can_gateisFalse, and every plan compilesinformational/proposed. Discovery proposes; only the existing host-owned approval path can license a gate, and the verdict still comes from the single Verdict Engine. - New domain-neutral built-ins. Context grounding, field substance, tool-choice validity and trajectory termination — each versioned, deterministic and diagnostic; absent evidence reads
non_evaluable, never a fabricated0.0. - The
discover-metricsloop. Rich trace acquisition, a semantic compiler from evidence to evidence-linked instruments, catalog execution, a metric template, a cross-workflow metric index and immutable bundle history.
v0.1.1 · 9 Aug 2026
Repository governance & CI
A governance and CI patch. No product runtime, public contract, authority or verdict change — the vendored runtime and all typed artifacts are byte-identical to v0.1.0.
- Added — a documented CI quality gate. The required checks that gate a merge into
main: static analysis (Ruff, mypy--strict, Bandit, core-isolation), tests on Python 3.12 and 3.13 with coverage, docs-consistency and public-surface guards, and the supply-chain / secret scans. SonarCloud stays informational. - Changed — branch protection.
mainenforces green-before-merge: the full check set must pass, the branch up to date, conversations resolved, linear history, and no direct or force pushes. - Fixed. A deterministic secret scan (excluding a false-positive detector), and a hardened Dependabot auto-merge.
v0.1.0 · 9 Aug 2026
Initial public release — pre-alpha
The first public release: a small, vendored, local-first evaluation framework for agentic LLM applications, delivered as a Claude Code / Codex plugin. Its governing rule is no false confidence — a fresh run is informational by design, real non-reference signal but not a quality pass.
- Effect-free Evaluation Core. Typed contracts, explicit status and validity, provenance, typed authority, baseline comparability and a single Verdict Engine.
- Runtime Harness. A CLI, a plan-before-effects control plane, a diagnostic-first dashboard, an immutable run series and a drift watcher.
- Optional lanes + installer. Opt-in, pinned, deletable Langfuse / Phoenix / LangSmith connectors and an OpenAI-compatible judge; conservative host discovery; vendoring with a manifest and lock; and a metric-catalog scaffolder emitting only
proposed/informational/uncalibratedassets — a dual-runtime plugin from one canonical skills tree, with an always-on honesty guardrail.
Every entry maps to a tag.
A release history padded with fake entries would be the exact false-maturity signal we refuse. Every version and date above comes straight from a GitHub release — no invented versions, no invented dates — and GitHub Releases stays the canonical source.
Related