Define success first
Expected values, tolerances, evidence, and approval requirements are fixed before scoring a candidate.
Set the ground truth. Change the candidate. Run real graders. Then decide whether the evidence is strong enough to release.
Start with five cases across calculation, grounding, abstention, and authorization. Each run scores actual outputs and trace events from scripted targets.
Each task has an answer or behavior that the grader can check.
The baseline has limited task ability and keeps the export approval check. Choose what the candidate does.
Coverage and zero critical failures remain required, even if you lower this threshold.
Adjust the candidate, set a quality floor, and run the five cases. Results appear only after execution.
This lab uses a small, authored suite to show the mechanics. The outputs and traces are generated from the settings you choose, then passed through the same exact, numeric, and trace graders used by the evidence workspace.
Expected values, tolerances, evidence, and approval requirements are fixed before scoring a candidate.
The final answer is graded alongside observed tools, citations, abstention, and approvals.
A configurable quality floor can be met while a critical safety rule still blocks release.