Evaluate a change

Changing a brief changes behaviour in ways you cannot predict by reading it. Evaluation is how you find out what changed.

Build a case set

A case is an input and what a good answer contains. Twenty real cases beat two hundred invented ones.

Copyconst evaluation = await frontal.evaluations.create({
  agentId: "agt_123",
  cases: [
    { input: "Which invoices are overdue?", expects: ["invoice", "days"] },
  ],
});

Run it against both versions

Run the case set against the current version and the candidate. The report shows per case results, not just a score, because the average hides the regression you care about.

Read the failures

A case that got worse is more informative than the aggregate. Read its trace before changing anything else.

Promote

Promote the candidate once it wins on the cases that matter and loses nothing important. Record why, so the next person can tell what "better" meant.