Runtime & Integrations · Evaluation Core
Run Evaluation
Composing the whole pipeline with zero effects.
run_evaluation(*, run_id, examples, evidence, plans, dimensions, baseline=None,
comparison_requested=False) → RunRecord is the one function a vendored host or test calls.
It runs the entire core pipeline and calls the Verdict Engine exactly once.
Everything else in the core is a stage; this is the composition. The Runtime Harness builds the
inputs — a MetricPlan per metric, the Example list, the
EvidenceBundle, and the provenance dimensions — and
run_evaluation turns them into a typed RunRecord with no I/O.
What it does, in order
- Scores each example per its
MetricPlan, then_validatedfails closed on an undeclared score name or an out-of-range value. - Computes comparability up front and injects the live
baseline_stateinto authority. - Resolves authority per metric and aggregates eligible scores.
- Calls
decide_verdictonce, and packs everything into theRunRecord.
python
from evalglass.core import run_evaluation
record = run_evaluation(
run_id="quickstart",
examples=examples, evidence=evidence,
plans=plans, dimensions=dimensions,
)
One verdict, computed once
Comparability is resolved before authority so a required-baseline gate sees the real state, and
the verdict is decided in a single call. There is no second place a run outcome can be formed.