“Set up EvalGlass in this repo.”
you getcandidate call sites discovered and the runtime vendored into your repo as plain code — nothing gates yet.
verbs: /evalglass setup → connect
Get the plugin
Install the EvalGlass plugin in your coding agent.
Install EvalGlass as a Claude Code plugin (Codex supported as a second runtime) and operate it through your coding agent — it grants no gate, approve, or certify authority of its own.
One install, then you drive it in plain language or the /evalglass verbs. A fresh
install gates nothing — you approve every threshold yourself.
Two commands
Claude Code + Codex
No keys · no telemetry
Apache-2.0 · pre-alpha
Install in Claude Code
Two commands — a marketplace, then the plugin
/plugin marketplace add Syntelesis-Lab/evalglass /plugin install evalglass@syntelesis-lab
Read it as plugin@marketplace: evalglass is the plugin; syntelesis-lab is the publisher's marketplace catalog (it matches the GitHub org and can hold more Syntelesis Lab plugins later). marketplace add still takes the GitHub repo slug. Prefer not to use the marketplace? Clone the repo and point Claude Code at it with claude --plugin-dir .
Then just ask your coding agent what you want:
Evaluate my agentic app with EvalGlass.
Prefer to read first? Browse the source on GitHub — Apache-2.0, and the runtime vendors into your repo as plain code you can read end to end.
See it work before you install anything
The bundled quickstart runs with no setup and no host repo — real, populated non-reference
signal and an honest informational verdict, so you can read a result before wiring
it into your own code.
/evalglass run --example quickstart
First things to say
Once it's installed, describe what you want — the right verb runs underneath. A few to start with:
“Evaluate my agentic app after the model switch — what improved, what regressed?”
you geta scorecard with per-metric deltas and an honest verdict — informational until you approve a gate, so nothing fails the build yet.
verbs: /evalglass run → view
“Explain why each number is or isn’t trustworthy.”
you getthe authority and diagnostics behind every metric — a missing field is reported as missing, never invented as 0.0.
verb: /evalglass explain
“Set up a CI gate for the checks we’ve reviewed and approved.”
you geta CI workflow that blocks only on a gate you approved — no auto-approval, no false green.
verb: /evalglass ci
Also supported — the direct CLI
The plugin is an additive convenience layer; the underlying framework still works as a plain CLI with no agent. If you already vendored EvalGlass this way, nothing changes — that path is unchanged and still supported.
Integration-time — vendor the runtime into your repo
python -m evalglass.installer install --root .
Run an evaluation against the vendored runtime
PYTHONPATH=evals python -m _evalglass.harness.cli run --config evals/evalglass.yaml
Host evaluation always runs the vendored _evalglass runtime under your repo — never the plugin — so it keeps working after the plugin and the agent are gone. Don't pipe an installer into your shell (curl … | sh): a tool whose whole value is auditability should never ask you to run un-reviewed code. Everything here is plain files you can read.
On Codex now
EvalGlass packages for Codex from the same repository and the same canonical skills tree — no forked guidance. Ask your agent to evaluate your app and the same umbrella skill routes the work. A public Codex marketplace listing is a maintainer step that isn't published yet; until then, the portable direct CLI above is the way in on Codex. How the second runtime works →
The plugin grants no authority.
Anything it scaffolds starts as a draft: a proposed dataset, an informational
metric, an uncalibrated judge. Your first run reports a Scorecard, but no score gates CI until
you approve a threshold in your own config — there is no gate verb to do it for you. The
vendored runtime keeps working after you remove the plugin, on either runtime.
How trust works →
Next
How it works
the journey, end to end
The /evalglass verbs
the whole surface, with honesty notes
How it's vendored
what setup writes, and what it never touches