Trust Model
Diagnostics
Why every result has a reason.
A Diagnostic is the structured reason attached to a non-perfect result —
code, severity, message, location,
details, cause, and evidence_refs. It is what makes a
blocked, errored, or below-target result actionable instead of mysterious.
A bare number tells you something went wrong but not what or where. EvalGlass attaches a
Diagnostic to every state that is not a clean pass, so the “why” travels with the result all
the way into the Scorecard. This is the companion rule to no
0.0: a measurement is never quietly low, and it is never quietly missing.
The fields
| Field | Carries |
|---|---|
code | A stable, machine-readable reason key (e.g. reference_missing). |
severity | One of info, warning, error. |
message | A short human-readable explanation. |
location | Where it happened — the example, unit, or field. |
details | Structured context (the metric, the offending value, expected shape). |
cause | An underlying error or nested diagnostic, when there is one. |
evidence_refs | Pointers to the evidence that produced the result. |
What one looks like
A reference metric with no gold answer is non_evaluable, and it says so:
json
{
"code": "reference_missing",
"severity": "warning",
"message": "No reference provided; exact-match is non_evaluable.",
"location": "example#42",
"details": { "metric": "exact_match" },
"cause": null,
"evidence_refs": []
}
Note what this is not: it is not a score of 0.0, and it is not an error in your
model. It is a precise statement that the metric could not apply here — which you can act on by
adding a reference or scoping the metric out.
Severity is not a verdict
A diagnostic’s
severity describes the finding, not the run outcome. A
warning does not by itself fail CI; only the Verdict
Engine decides a run outcome, from measurement and authority.