Regression run · v0.3
Eight synthetic scenarios covering normal, ambiguous, sensitive, and escalation paths.
| Scenario | Intent | Risk | Score | Decision |
|---|
Release rubric
Weighted for correctness and safe escalation—not surface-level fluency.
Go / no-go rule
Release only when the mean scenario score is at least 90, every required scenario is covered, and no blockers remain. The scenario scores illustrate how a reviewer might apply the rubric.
RELEASE DECISION · 01 SEP 2026
Hold v0.3; fix escalation and unsupported-claim behavior
The agent is clear and on-tone in routine interactions, but two failures create unacceptable operational risk. Shipping now would trade speed for preventable rework and customer distrust.
What the run says
- 6 of 8 scenarios pass.
- Routine status and scheduling flows are reliable.
- A missing-record case triggers an unsupported certainty claim.
- A time-sensitive medical-record request fails to escalate.
Smallest safe fix
- Add an evidence-required rule before status claims.
- Route urgent or ambiguous medical-record requests to a human queue.
- Rerun the two failures plus adjacent edge cases.
- Ship only after blocker count reaches zero.
Owner: Product OperationsReviewers: PM · Engineering · Customer SuccessRecheck: after patch