Start here

Install the plugin

Adopt EvalGlass in your own repo, then run.

EvalGlass installs as a Claude Code plugin: it vendors the framework into your repo and scaffolds host-owned starter assets, so you can run a first evaluation against your project. Like the bundled sample, that first run is informational by design — nothing is validated yet.

This is the adoption path. Installing never overwrites your files, and authority starts empty so no gate fires until you say so. If you only want to see a Scorecard first, run the zero-install demo /evalglass run --example quickstart — it produces real non-reference signal and an honest informational verdict.

Two ways to start Just want to see a Scorecard immediately, with no install? Use the bundled Quickstart. This page is for putting EvalGlass into your own repository.

1 · Install the plugin

Add the marketplace (the GitHub repo slug), then install the plugin from it. These two commands are byte-identical wherever you run them:

bash

/plugin marketplace add Syntelesis-Lab/evalglass
/plugin install evalglass@syntelesis-lab

The install reads as plugin@marketplace: evalglass is the plugin, syntelesis-lab the publisher's marketplace catalog (matching the GitHub org). The plugin then drives the /evalglass verbs — see the CLI reference. Codex is supported as a second runtime; its public marketplace listing is next, not yet published.

Also supported: the direct CLI Not on Claude Code? The CLI path is additive and fully supported. Vendor the runtime with python -m evalglass.installer install --root ., which writes the same managed runtime and starter assets described below.

2 · What install writes

Install vendors the managed runtime into evals/_evalglass/ (imports rewritten to _evalglass, version pinned, with a manifest + lock) and scaffolds seven host-owned starter files — skipping any that already exist. The JSON summary lists what was scaffolded vs preserved:

json

{
  "managed_root": "evals/_evalglass",
  "framework_version": "0.1.0 (pre-alpha, untagged)",
  "scaffolded": [
    "evals/evalglass.yaml",
    "evals/datasets/sample.jsonl",
    "evals/traces/sample.jsonl",
    "evals/evaluators/answer_nonempty.py",
    "evals/authority.json",
    "evals/README.md",
    "evals/ci/github-actions.yml"
  ],
  "preserved": []
}

3 · Run

Run a first evaluation against your repo. Through the plugin that is /evalglass run; with the direct CLI it is the vendored _evalglass runtime:

bash

PYTHONPATH=evals python -m _evalglass.harness.cli run \
  --config evals/evalglass.yaml

You get a first informational Scorecard for your repo, exiting 0. It is informational because the scaffolded gold is proposed, no threshold is approved, and evals/authority.json is empty. That is the honest default — read it with Read a Scorecard, then make a gate real with Promote a Gate.

What you now own Everything under evals/ except _evalglass/ is yours to edit and validate. The runtime works even after the plugin and any coding-agent context are removed — it never imports the installer. Upgrades go through Revendor Safely.

Next steps

Run and read

Read a Scorecard

Understand your first result.

Ship

Promote a Gate

Validate gold, approve a threshold, gate.

Vendoring

EvalGlass Skill

How discovery, vendoring, and scaffolding work.