Evaluating an AI Feature with a Repeatable Test Set

What you will learn

Start with one user task, its expected behavior and the harm of a wrong answer. Build a small, versioned set of synthetic examples that covers normal requests, missing context, conflicting evidence and out-of-scope requests. Use exact checks for requirements that have a clear answer and human review for qualities that need judgment.

Worked example

The reviewed example ties assertions to a requirement, input or explicit risk.

Try it

Create ten synthetic examples for one behavior. Write the expected criteria first, include missing-context and conflicting-evidence cases, compare two versions on the same inputs, then inspect disagreements and explain the tradeoffs. Keep the dataset and system versions with the results.

Review checklist

• Define the user task, expected behavior and risk.
• Version the examples and expected criteria.
• Separate exact checks from human-reviewed judgments.
• Review failures and reviewer disagreement; do not hide them in one score.
• Record model, prompt, retrieval and dataset versions.
• Keep the workflow vendor-neutral and check current product docs before choosing a tool.

• NIST AI Risk Management Framework (AI RMF 1.0)
• NIST AI RMF Generative AI Profile
• OpenAI Evaluation Best Practices