Project-specific checks for agentic apps

When you set out to evaluate an agentic application, the first instinct is usually to find a benchmark and report a number. That instinct comes from a world where the program does the same thing every time and the only open question is general capability. An agentic app does not live in that world. It selects tools, takes multi-step trajectories, calls into your own systems, and reaches outcomes that depend on your product's rules — rules that exist nowhere but your codebase. The number a packaged benchmark hands back measures a generic capability; it cannot measure whether your agent did what your product needs. The useful asset is therefore not a score on someone else's test. It is a set of checks that encode the behavior your application must perform, authored around your real traces and owned in your repo. This article is about how that asset is built, who owns each part of it, and why no off-the-shelf benchmark can build it for you.

The asset is your checks, not your score

A project-specific check is a small, repeatable eval that asks one question about your application's behavior and answers it the same way every time you run it. The question is drawn from the product, not from a leaderboard: did the agent select the right tool for this kind of request? did it follow the required steps, in order, before acting? did the final answer stay grounded in the documents it retrieved? Each of these is a behavior your product depends on, and each is invisible to a benchmark that never knew your product existed. The reason agentic behavior demands this kind of authored check, rather than a generic pass/fail, is that the thing being graded is a trajectory, not a single output — the end-to-end log of decisions, tool calls, and reasoning steps. A right answer reached by calling the wrong tool and recovering by luck is not the behavior you want to ship, and only a check that inspects the trajectory can tell the difference. EvalGlass scopes a check to a declared slice of that trajectory — a single call, a step, a whole trajectory, or a session — so the question stays precise.

The evidence those checks run on is your own behavior, captured as traces and turned into examples. EvalGlass imports exported traces — local JSONL, OpenTelemetry or OpenInference — with no SDK and no network call on the run path; live pulls from a tracing platform are an experimental, opt-in lane (the connect --live verb). From those traces you pull the turns that matter and shape them into evaluator-ready examples: an input, an output, the unit it belongs to, and an optional reference. This is where the work pays off. The same incidents that taught you what your product must do — the refund that should never have fired, the retrieval that drifted off-topic — become the examples your checks measure against, run after run. A benchmark cannot offer you this, because it has none of your history.

The deliverable is not a number you can quote. It is a set of checks that encode what your application must do — owned in your repo, run on your own behavior, every time something changes.

Why a packaged benchmark can't do this

Benchmarks are genuinely useful for comparing general capability across models, and nothing here argues for abandoning them. The argument is narrower and more practical: a benchmark is structurally unable to test the behavior your product depends on, and leaning on one alone is riskier than the comforting score suggests. The first limit is the obvious one. Your policy, your tools, and your edge cases live in your product, not in any packaged dataset. The agent that scores well on a public tool-use benchmark can still call your refund tool on a request that was only a question, because the rule "never refund without a manager flag" appears in no benchmark anywhere. The behavior that matters to you was simply never tested.

The second limit is that the score itself can be unreliable in ways that are easy to miss. A 2024 survey documents how benchmark data contamination inflates results when test data leaks into a model's training set: the model has, in effect, seen the answers, and the headline number rewards memorization rather than capability.¹ The third limit is sharper still. A 2026 audit of agent benchmarks reported that agents could reach near-perfect scores "without solving a single task," by exploiting the harness rather than performing the intended work.² A number an agent can raise without doing the job is not authority — it is the opposite, a confident green that means nothing. These are field findings about benchmarks, not results EvalGlass produces; EvalGlass does not solve contamination or gaming, and it does not replace benchmarks. What it does is let you add the checks a benchmark structurally cannot provide: ones written against your behavior, on examples you control.

The anatomy of a check, and who owns each part

A check is not one artifact but several, and the discipline that keeps it honest is knowing which parts your coding agent may draft and which parts only you may confer. Your agent — Claude Code as the primary runtime, Codex supported with a public listing still to come — can scaffold most of a check faster than you would by hand. But the parts that grant authority stay with the host: you, and the reviewed records in your repo. The agent operates; it does not authorize its own work. That separation is EvalGlass's own design choice, the same "no self-approval" principle that runs through the rest of the tool.

Five parts make up a check. The examples are the evaluator-ready items — input, output, unit, optional reference — pulled from your exported traces or hand-authored; they are host-owned. The evidence is the datasets and traces those checks run on, imported locally with no SDK and no network. The metric or rubric is where the measurement is defined: a MetricSpec declares what a metric means and which lens it uses, while a judge rubric encodes the criteria an LLM-as-judge grades against. Your agent can scaffold both, but they arrive as drafts. The threshold and baseline are the pass boundary and the run to compare against — both host-owned, and a delta only counts as a regression when the two runs are comparable. Finally the authority record is the typed record that decides whether a metric may gate; only a host record grants it. The table below sets out each part, its owner, and the state it sits in until you deliberately promote it.

The two checks worth understanding in detail are the metric and the rubric, because they are where "what your product must do" gets written down precisely. A metric can take a reference lens, comparing the output to validated reference data, or a non-reference lens, inspecting the output alone — useful when the right answer is "the refund tool was not called," which needs no validated reference answer to judge. When a built-in metric won't capture your rule, you write a host evaluator: a small function in evals/evaluators/ that receives the example and the evidence as plain data and returns a score. A judge rubric is the path when the criterion is qualitative — was the answer grounded, was the tone appropriate — and it carries an extra obligation, calibration, because an uncalibrated judge is an opinion, not a measurement. The full vocabulary lives in metrics and judge metrics; the Metric Builder extension scaffolds a proposed metric for you to review.

Proposed versus validated, informational versus gating

Two distinctions do most of the load-bearing work in keeping a check honest, and they are easy to conflate. The first is proposed versus validated, and it applies to evidence. Anything your agent scaffolds — examples it pulled from traces, a starter dataset it assembled — lands as proposed: present and usable, but not yet domain truth. Only a host can move a dataset to validated, and that act is a deliberate review, not a side effect of generating the data. The governance here is strict on purpose: any generated or synthetic case is forced to proposed and can never enter as validated reference data, no matter what produced it. The distinction that matters, then, is not "example versus dataset" but where a row came from and whether a human signed off on it.

The second distinction is informational versus gating, and it applies to metrics. A scaffolded metric lands informational: it reports a value and a delta, but it never fails a build. A metric becomes a gate only when four host-owned conditions all hold — a validated dataset, an approved threshold, the metric explicitly set to gating, and, for a judge metric, a calibrated judge. Crucially, there is no promote, gate, approve, or certify verb your agent can call to do this. Promotion is a host-owned edit you make in your own config — setting the dataset to validated, the threshold to approved, and the metric to gating. The agent may type that edit at your explicit direction, but it cannot grant the authority behind it. This is why a fresh EvalGlass setup gates nothing and exits cleanly: the tool refuses to manufacture confidence from data no human has reviewed. The four conditions, and the exact YAML edit that clears each one, are on promote a gate.

1. 01 · behavior

   What your agent must do

   Tool selection, trajectory, outcome, policy — scoped to a declared slice (call, step, trajectory, session).

2. 02 · evidence

   Traces & examples

   Exported traces and host-owned examples. They land proposed until you review them.

3. 03 · check

   Metric or rubric

   Scaffolded by your agent as proposed / informational — it reports, it can't gate.

4. 04 · authority

   Your review point
   • informational — the default; reports, never fails CI.
   • blocked — a gate can't be claimed honestly yet.
   • can_gate — you validated data, approved a threshold, set it gating.

One check, end to end

Make this concrete with a single rule. Suppose your support agent scores well on a public tool-use benchmark, but you keep noticing it calls the refund tool when a customer only asked a question — a clear violation of your policy that a refund needs a manager flag first. No benchmark tests this, because the rule is yours. So you author a check for exactly it, and you watch where the agent's reach stops and yours begins.

You start in natural language. You ask your coding agent to pull the refund-tool turns from your exported traces and build a check that the agent never calls refund without a manager flag, and to leave it proposed. The agent reads the traces, shapes the relevant turns into examples, and writes a host evaluator into evals/evaluators/ that inspects whether the refund call carried the manager-flag field. The evaluator is written to return non_evaluable when a trace lacks that field rather than a false 0.0 — because "we couldn't measure this case" and "this case scored zero" are different claims, and collapsing them is exactly the kind of false confidence the tool refuses. Everything the agent produced lands proposed and informational.

Now you run it. The vendored runtime evaluates the behavior locally — no platform, no keys, no network — and returns a bounded scorecard: how many turns violated the policy, how many were non-evaluable, and the delta against a prior comparable run. Read the scorecard and you learn something real about your product. But notice what has and has not happened. The agent created the evidence and the metric; it did not authorize them. The check is reporting, not deciding. To make it a gate you do the host work yourself: review the pulled examples and validate the dataset, approve a threshold (zero violations on these examples), set the metric to gating, and — if you had used a judge rather than a deterministic evaluator — calibrate it. Only then does the check's authority resolve to can_gate and a future violation can fail CI. Even fully gating, the claim it makes is bounded: "no refund-without-flag on the examples you supplied." It does not certify that the policy is never violated anywhere — only that it held on what you measured.

Sources

The limits of generic benchmarks below are field context, not claims about EvalGlass — and EvalGlass does not solve benchmark gaming or replace benchmarks.

1. Xu et al. — Benchmark Data Contamination of Large Language Models: A Survey (2024): test data leaking into training inflates and undermines benchmark scores.
2. Wang et al. — Do Androids Dream of Breaking the Game? (BenchJack) (2026): agents can reach near-perfect benchmark scores without performing the intended task.