Runtime & Integrations · Evaluation Core

Provenance & Baselines

When may EvalGlass claim a regression?

fingerprint_dimension() hashes each provenance dimension (SHA-256 of canonical JSON). RunFingerprint records ten of them; a ComparableRunFingerprint compares two runs and resolves a BaselineState. A regression claim is only made when the gating dimensions match.

A regression without comparability is not a claim. So the core computes comparability as a typed state before any delta — the code behind Provenance and Baseline Comparability.

Dimensions

Ten dimensions are recorded (REQUIRED_DIMENSIONS): framework, metric_spec, evaluator, dataset, example, evidence, config, policy, authority, baseline. Six of them gate a comparison (DEFAULT_GATING_DIMENSIONS): framework, metric_spec, evaluator, dataset, config, policy — example is deliberately not gating, so different examples still compare.

Fail-closed comparison

fingerprint_dimension raises on a non-JSON value (e.g. a set) rather than guessing a hash.
changed_dimensions treats an absent dimension as changed, not as a match.
So a fingerprint can only under-claim comparability, never over-claim it.

The full fingerprint lands in RunRecord.provenance and the resolved comparability in RunRecord.comparable — both recomputable by a reviewer or a later run.

Promotion is explicit A baseline is only written by an explicit command (baseline update), never by an ordinary run. Comparing against it is opt-in; with no request the state is comparison_not_requested.

Next steps

Evaluation Core

Authority Resolution

How a required, un-comparable baseline blocks a gate.

Trust Model

Baseline Comparability

The four states.

Guide

Compare Baselines

Run a comparison the honest way.