Use case · Trace analysis
You have traces. Turn them into checks.
Traces are evidence waiting to become checks. Ask your coding agent to read your exported traces, author evaluation cases from them, and cluster the most common failure modes — so the behavior you already recorded becomes something you can measure.
Create evaluation cases from our traces and show the most common failure modes.
What it checks
import of exported OpenTelemetry / OpenInference / local-JSON traces → authored cases · failure clustering. The import is shipped (now); a live pull from Langfuse / Phoenix / LangSmith is an opt-in lane reachable with the connect --live verb experimental — off the required path, a pull is proposed and cannot gate. the typed on-scorecard cluster view (Scorecard.clusters) groups failing and non-scored cases by shared Diagnostic.code next — a non-scored case is grouped by its cause, never written as 0.0; a browsable metrics explorer is a separate, still-planned view on a different axis. Authored cases land proposed.
The scorecard
Verdict informational — cases are proposed from exported traces, not validated. Illustrative example, not a measured result.
The cluster rows above sketch the typed Scorecard.clusters view
(next) — failing and non-scored cases grouped by cause,
shown informationally, not a field executing on a measured run.
What it will not claim
Importing an export is not a live integration. EvalGlass reads the traces you exported — no SDK, no network — and authors proposed cases from them. A live pull is an experimental, opt-in lane (the connect --live verb; credentials are env-var names, data_policy defaults to unknown, and a pull is proposed and cannot gate), never production live tracing; a browsable metrics explorer stays planned — neither is shown as shipped. No false green →
Ask your coding agent.
Evaluate my agentic app using EvalGlass.
Related