Build checks
Traces
Import exported behavior — locally, no SDK, no network.
connect --live verb) —
see Connectors — and is not what this page does.
“OpenTelemetry support” here means EvalGlass can map an export you already have, not that it connects to your collector. The SDK never enters the picture: you hand it a file, it converts the spans at the edge, and vendor objects never travel inward.
The formats it reads
traces[].format | Source | Notes |
|---|---|---|
local | EvalGlass JSONL | Recorded behavior in the native shape. |
opentelemetry | Exported OTel span JSON | Dict spans mapped at the edge — no SDK. |
openinference | Exported OpenInference span JSON | Same mapper, OpenInference conventions. |
How a trace becomes evaluable
Each source converges through the same typed spine — TraceEnvelope (vendor-neutral,
with metadata, data policy, and provenance) → EvalUnit (the declared slice:
call / step / trajectory / session) →
Example. A span missing an id or output does not become a low score:
- exported · mapped locally
- evaluable
Example reaches the metrics. A live pull is a separate,
experimental, opt-in lane (connect --live; see Connectors).trace_mapping_incomplete — a typed
diagnostic, counted and excluded — never a
fabricated 0.0. The record reconstructs faithfully or refuses; it never silently
shrinks.
Grade the whole run: the unit: key
By default a route grades one model call at a time. A traces[] route can
instead declare unit: trajectory | session | step to group every call sharing a
trace_id into one aggregate and grade the whole agent run
(next; built in the framework, not yet in the version the plugin
installs). Absent unit: is byte-identical to call.
yaml
traces:
- path: runs/agent.jsonl
unit: trajectory # grade the whole run, not one call
The aggregate is scored by the built-in trajectory_shape@1; its egress is the
worst of its members (one forbidden member fails
the whole aggregate closed), and an empty trajectory is
non_evaluable, never a fabricated 0.0. Because trace
data carries no validated gold, the result stays informational and
cannot gate — an aggregate green is still not proof the agent succeeded. Full walkthrough:
Trajectory & session grading.
What this is not
This page is exported-import only. Pulling from a live Langfuse / Phoenix / LangSmith backend, or pushing scorecards to a hosted dashboard, are separate opt-in lanes with their own, more conservative status — Connectors documents each, badged per cell.