Your build starts here
Replace bracketed details with your own goal, context, and checks. Keep the first build small enough to inspect.
Help define evaluation material for [reusable prompt task]. The task rules are [rules], and existing development examples are [examples]. The prompt will later be revised using development feedback.
Propose development cases for known ambiguities and a separately identified final set covering normal, edge, empty, unsupported, and misleading near-miss inputs. Give each case a stable identifier and expected outcome or rubric. Avoid simply changing names in the demonstrations and calling the result independent coverage.
Explain how a human maintainer should freeze and store the final set outside the material used for prompt revision. Do not reuse this conversation or its final-case answers as context for the later prompt-revision assistant. If a case's expected result is genuinely ambiguous, flag it for adjudication instead of silently choosing a convenient answer.
Return the two sets, their intended roles, and a rule for promoting discovered failures into future development data. State that repeated tuning against final cases changes their role; the set name alone does not preserve independence.Make this prompt yours
Use to organize a workflow, not to claim statistical independence merely from separate filenames. Human review should check whether cases represent real use. Keep the final cases out of the revision assistant’s conversation as well as its files.
You might also hear
- held-out prompt evaluation
- test leakage prevention
- final prompt cases
The docs behind it
These references explain the underlying concepts and APIs. The prompt templates and project plans are original examples to adapt to your own setup.
Research sources reviewed .
Editorial review · October 28, 2026
Supported: Original, unexecuted test-design template. It explicitly keeps final cases outside the assistant's prompt-revision conversation as well as development files. This reduces leakage; it does not certify representative coverage.