Build checks · non-goal
Synthetic scenarios
Generation is a non-goal. The governance is already real.
Synthetic scenario generation is planned and opt-in — there is no
generator today. A Core scenario generator is planned; Intelligence offers a commercial
targeted scenario search. What ships now is the governance that bounds either: any
generated or synthetic case is forced to proposed. It can be recorded and used as evidence, but it
can never enter as validated gold or self-approve a gate. This updates the site's earlier stance
that generation was ruled out, and is recorded in an accepted public ADR
(the ADR process →).
Generated evidence is the most tempting place to launder confidence — a model writes gold answers and a gate suddenly looks justified. EvalGlass closes that path before the generator exists: the most a generator could ever do is propose, and a human still disposes.
The governance that already holds
| Artifact | What it can claim | What grants authority |
|---|---|---|
| Synthetic dataset | Status is always proposed — a hopeful declared_status is accepted but never honored. | A host validation that re-issues it as validated. |
| Benchmark result | Provides evidence toward a threshold — evidence only. | A host ApprovedThreshold; asking a benchmark to approve one raises GovernanceError. |
These are enforced in the library, not by convention:
import_synthetic_dataset(..., declared_status="validated") still returns a
proposed dataset.
Planned — and never shown executing
Because no generator ships, you will not see one produce gold anywhere in these docs. When it
lands it will be opt-in, and every case it writes will arrive
proposed, reviewable before it can back a gate. Track status on
the roadmap register.