Case sets and promotions
Build representative cases, compare versions, and make promotion decisions from evidence rather than intuition.
A case set is a collection of real inputs, expected outcomes, and criteria that represent the work your system must do. Include ordinary cases, edge cases, missing-context cases, and situations where the correct result is to refuse or request approval.
Run the same set against the production and candidate versions. Review regressions individually; an aggregate score can hide the failure that matters most. Record the approver, evidence, and reason for promotion so an operator can understand why a version reached production.
Updated Aug 31, 2026