Use case · Feature testing
A new AI workflow ships. Test it before launch.
A new agentic workflow has no track record. Ask your coding agent to scaffold checks and scenarios for it so you can see how it behaves before release — knowing that scaffolded checks arrive provisional, for you to review.
Build checks for this new AI workflow before we release it.
What it checks
new metrics and rubrics for the workflow · scenarios you author from the spec. Scaffolded checks land proposed · informational — they don’t gate until you review them. Automated scenario generation is a decided non-goal non-goal — no generator ships, by design; you author scenarios from your spec. A “launch scorecard” is evidence, never approval-to-launch.
Discover what to measure next
Before you can test a new workflow, you need the right checks. The
discovering-metrics skill deep-reads your repo, prompts, output shapes, and exported
traces to propose a typed MetricCatalog — anchored to the ways your app actually fails,
across an eleven-dimension quality taxonomy and three tiers (deterministic, reference,
judge) — and scaffolds it into the canonical evals/ layout. Everything it proposes lands
proposed / uncalibrated and informational: it suggests what to
measure, it never approves it. This is what to measure, not scenario generation.
The scorecard
Verdict informational — every check is proposed and ungated. Illustrative example, not a measured result.
What it will not claim
A launch scorecard is not a launch approval. Scaffolded checks are proposed until you validate them, scenario generation is a decided non-goal — no generator ships, by design — and EvalGlass authors cases from your spec, it does not manufacture reference data. No false green →
Ask your coding agent.
Evaluate my agentic app using EvalGlass.
Related