Run and read

Run an evaluation

One verb turns your evidence into a typed verdict.

/evalglass run executes the evaluation against your vendored runtime and reads back the typed verdict — no network, no keys. The plugin only issues the command and reads the artifacts; it never computes the verdict itself. A fresh run is informational by design, because nothing is gated until you approve it.

A run reads the evidence you own, applies the metrics you declared, and lets the single Verdict Engine decide the outcome — then emits a Scorecard, a RunRecord, and a report. It measures and reports; you decide and gate.

Run it

Through the plugin it is one command; with the direct CLI it is the vendored _evalglass runtime. Both produce identical artifacts:

bash

# Plugin (Claude Code)
/evalglass run

# Direct CLI — the vendored runtime in your repo
PYTHONPATH=evals python -m _evalglass.harness.cli run \
  --config evals/evalglass.yaml [--out <dir>] [--format terminal|ci]

Before you adopt anything, /evalglass run --example quickstart runs the bundled sample — real, populated non-reference signal and an honest informational verdict. See the Quickstart and Install for the first-run paths; the full flag surface is in the CLI reference.

What a run produces

Every run writes three artifacts under the reports directory — the JSON is authoritative:

Artifact	What it is
`scorecard.json`	The primary summary: verdict, `ci_should_fail`, per-metric values and status counts, authority, baseline state, diagnostics. Contract →
`runrecord.json`	The complete, reproducible record: every Score, provenance fingerprint, and comparability result. Contract →
`report.md`	A human-readable rendering of the typed data — never the source of authority. `--format ci` renders GitHub annotations from the same verdict.

Where the verdict comes from

A run is observe → measure → decide: it reads host-owned evidence (your dataset and exported traces — no live pulls), applies each metric, and the Verdict Engine resolves one typed outcome. Authority decides whether any gate was even active; on a fresh run none is, so the run is informational and exits 0 — measured, but gating nothing.

A first run gates nothing — and that's correct Until you validate a dataset and approve a threshold, every metric is informational: the run reports values but enforces nothing. To make a metric actually gate CI, see Promote a Gate. Read the result you just produced with Read a Scorecard.

Next steps

Run and read

Read a Scorecard

Every field in the result, and what it can't prove.

Run and read

Explain a result

Why a number is — or isn't — trustworthy.

Reference

Verdicts & exit codes

How the one outcome is decided.