Build checks · non-goal
Synthetic scenarios
Generation is a non-goal. The governance is already real.
Generating synthetic scenarios is a non-goal — there is no
generator today. What ships now is the governance that will bound one: any
generated or synthetic case is forced to proposed. It can be
recorded and used as evidence, but it can never enter as validated gold or self-approve a gate.
Generated evidence is the most tempting place to launder confidence — a model writes gold answers and a gate suddenly looks justified. EvalGlass closes that path before the generator exists: the most a generator could ever do is propose, and a human still disposes.
The governance that already holds
| Artifact | What it can claim | What grants authority |
|---|---|---|
| Synthetic dataset | Status is always proposed — a hopeful declared_status is accepted but never honored. | A host validation that re-issues it as validated. |
| Benchmark result | Provides evidence toward a threshold — evidence only. | A host ApprovedThreshold; asking a benchmark to approve one raises GovernanceError. |
These are enforced in the library, not by convention:
import_synthetic_dataset(..., declared_status="validated") still returns a
proposed dataset.
Planned — and never shown executing
Because no generator ships, you will not see one produce gold anywhere in these docs. When it
lands it will be opt-in, and every case it writes will arrive
proposed, reviewable before it can back a gate. Track status on
the roadmap register.