Ship
Local-first workflow
It runs on your machine; your repo stays the source of truth.
EvalGlass vendors a managed runtime into evals/_evalglass/
and leaves everything else under evals/ host-owned. There
is no platform to pay for, no provider keys to buy, and no telemetry leaving your repo. The runtime
even outlives the tooling that installed it. This is the operational companion to the marketing
Local-first page.
“Local-first” is not just where the code runs — it is who owns the truth. Authority lives in host-owned records, not in the framework, so an upgrade can never quietly change a gate.
Managed vs host-owned
| Path | Owner | What lives there |
|---|---|---|
| evals/_evalglass/ | Managed | The vendored core, harness, and adapters — replaced only by a deliberate revendor. |
| evals/datasets/ · traces/ · evaluators/ | Host-owned | Your examples, exported traces, and host evaluators. |
| evals/authority.json · thresholds · calibration · baselines | Host-owned | The records that confer authority — what makes a gate real. |
| evals/evalglass.yaml | Host-owned | Your run config, referencing the records above. |
MANAGED
evals/_evalglass/ · sealed
core · harness · adapters
replaced only by revendor
grants no authority
never overwrites
HOST-OWNED
everything else under evals/ · yours
datasets · traces · evaluators
authority.json · thresholds · calibration · baselines
evalglass.yaml
- managed (vendored, sealed)
- host-owned (authority lives here)
The three honest “no”s
| No… | Meaning |
|---|---|
| No platform to pay for | The runtime is vendored and runs locally; there is no hosted service in the loop. |
| No API keys to buy | The required path uses a deterministic fake judge — live models are an opt-in lane you own. |
| No telemetry leaving your repo | Nothing phones home; outward sinks are opt-in and point where you choose. |
The runtime outlives the tooling
Host evaluation runs the vendored
_evalglass runtime, never the installer and never
the plugin — so removing the plugin and any coding-agent context leaves every host verdict
byte-identical. The deletion-invariant holds across runtimes, including Codex.