Local-first
It runs on your machine. Your repo stays the source of truth.
EvalGlass Core is local-first: the vendored runtime runs on your machine and in CI, your repo stays the source of truth, and nothing is sent to a hosted EvalGlass service. Discovery and Intelligence are separate, opt-in products.
EvalGlass Core runs entirely on your machine and in your CI: no hosted tier, no keys to buy, and nothing is sent to a hosted service. Local-first isn’t a feature — it’s the design. (Discovery and Intelligence are separate, opt-in products — see below.)
The three honest “no”s — about Core
on your machine & repo
- datasets, traces & prompts
- the vendored runtime (
evals/_evalglass/) - scorecards & RunRecords
- thresholds, calibration & authority
never leaves it
- no telemetry — Core sends nothing home
- no provider keys on the run path
- no hosted platform or dashboard
- No platform to pay for. Core is not a hosted service, a dashboard, or a SaaS. Core evaluation happens in your repo, against your evidence.
- No API keys to buy. Core's required path needs no provider SDKs and no model keys. Live models are an optional, opt-in lane you own — not a toll on the core.
- No telemetry leaving your repo. Core collects nothing and sends nothing on its required path. Your datasets, traces, and scorecards stay where you put them.
Why local-first matters
There are no servers to keep running on the project’s behalf. A tool that runs entirely from vendored source in your repo does not switch off when a bill goes unpaid — it keeps working as long as your machine and your CI do.
What runs where — who owns the truth
- EvalGlass owns the effect-free Evaluation Core, the runtime harness, the installer, and the vendored runtime under
evals/_evalglass/— plain code you can read. - You own your datasets, rubrics, thresholds, calibration evidence, baselines, host-evaluator code, and your data policy. The agent populates host-owned files; it never fills in authority.
- Nothing is silently overwritten. Re-vendoring replaces the managed runtime; your host-owned truth is left untouched.
Where the family runs
“Local-first” describes Core. The commercial products are separate and bounded — adopting one is a deliberate, opt-in step, not something that happens when you install Core:
- EvalGlass Intelligence can be customer-invoked and run locally or in a private environment — you choose the application and the evidence window. Private preview.
- EvalGlass Discovery reads a bounded representation of your application — code, prompts, tools, selected traces — locally or in your own VPC, and proposes the evaluations it’s missing for you to review. Private preview.
- Adopting either changes nothing about Core. It does not change Core's licence or its local independence — Core keeps running on your machine, with or without the paid products.
The runtime outlives the tooling.
Because authority lives in host-owned files and the managed runtime is plain vendored code, the evaluation keeps working after the plugin and any coding-agent context are removed. Local-first means Core has no cost centre to monetise and no place to insert a meter — EvalGlass Core measures what your evidence supports and reports only what it can honestly claim. Host ownership, in detail →
Related