Runtime & Integrations · Evaluation Core
Evaluation Core
The effect-free centre that turns evidence into a verdict.
The Evaluation Core is the standard-library-only layer that holds all
meaning: contracts, score semantics, aggregation, provenance, authority, and the Verdict
Engine. It performs no effects — no file or network I/O, no clocks, randomness,
subprocesses, or model calls.
Keeping meaning effect-free is what lets a green result never outrun its earned authority. The
core receives typed data the Runtime Harness already
collected, and returns a typed RunRecord. It cannot reach out to inflate a score,
and nothing downstream can recompute its verdict.
EVALUATION CORE · EFFECT-FREE · STDLIB ONLY
no file / network I/O · no clocks · no randomness · no model calls
NORMALIZE
TraceEnvelope
EvalUnit
Example
measure
MEASURE
MetricSpec + EvidenceBundle
Evaluator
Score
reduce
AGGREGATE & QUALIFY
AggregatedMetric
Provenance / Baseline
ResolvedAuthority
decide
DECIDE & EMIT
Verdict Engine
RunRecord + Scorecard
Verdict Engine decides, and no I/O, network, clocks, randomness, or model calls cross the boundary.What the core owns
| Stage | Page |
|---|---|
| Input shapes & fail-closed parsing | Core Contracts |
| Value vs meaning of a result | Score Semantics |
| Choosing, declaring, implementing metrics | Metrics |
| Honest reduction of per-example scores | Aggregation |
| Fingerprints & regression claims | Provenance & Baselines |
| May this metric gate? | Authority Resolution |
| The single verdict path | The Verdict Engine |
| Composing the whole pipeline | Run Evaluation |
| The primary artifacts | Scorecard · RunRecord |
Allowed tools
The core uses only dataclasses, enum, typing,
json, hashlib, and (where exact threshold arithmetic needs it)
decimal. No third-party validation, config, or reporting libraries; no vendor SDKs.
Isolation is enforced in CI by tools/check_core_isolation.py.
Use it directly
A vendored host or test calls
from evalglass.core import run_evaluation. The Runtime
Harness exists only to build that function’s inputs (plans, examples, evidence, dimensions) and
to act on its output.