Prompt Versioning: Versions, Labels and Rollback

Prompts behave like code: a one-word change can fix a bug or break a hundred cases. Yet many teams still keep them as strings in application code, edited in place, with no record of which wording produced which result. This article describes the version-and-label pattern most prompt tools have converged on, and how to tie it to evaluation. It belongs to the LLM evaluation guide.

The pattern: immutable versions, movable labels

• A prompt has a stable name, such as support.answer.
• Each save creates a new version (v1, v2, v3) that never changes. Store the system prompt, the template with its {{variables}}, a commit message and a content hash.
• Labels such as staging and production point at one version each. Moving a label is the only way to change what an application receives.
• The application fetches support.answer at label production at runtime, with caching.
• Rollback moves the label back to the previous version.

Step by step

1. Move each prompt out of application code into the registry, as version 1, and point production at it.
2. Change application code to resolve the prompt by name and label, with a short cache and a safe fallback if the registry is unreachable.
3. Make every edit a new version with a short commit message saying what changed and why.
4. Point staging at the new version and run your evaluation dataset against it; compare with the production version case by case.
5. Promote by moving the production label only when the comparison passes. Record which run justified the move.
6. If production quality drops, roll the label back and confirm that new traces carry the old version number.

# Resolve the production prompt at runtime (NexGen QA example)
curl -H "X-Api-Key: $NEXGEN_API_KEY" \
  "https://app.qa-automation.com/api/v1/prompts/resolve/support.answer?label=production"

Review diffs, not just outputs

Show a line diff between versions, including added and removed variables, in review. Many regressions are visible in the diff itself: a dropped constraint, a variable renamed in the template but not in the code, an instruction moved below a long example.

Common mistakes

• Editing a version in place, so old evaluation results no longer describe any prompt that exists.
• Hard-coding a version number in application code, which turns every promotion back into a deploy.
• Promoting on a handful of manual tries instead of a dataset run.
• Not logging the prompt version on each trace, so production issues cannot be traced to a change.

Prompt management in NexGen QA

NexGen QA's Prompts module stores immutable versions with commit messages, moves staging and production labels with one-step rollback to the previous version, shows a diff between any two versions including variable changes, and has a playground that runs a version on your own provider key with tokens, latency and estimated cost. Applications fetch a prompt by name and label with any organization API key; responses carry an ETag and a 60-second cache lifetime. Free includes 10 prompts; Pro has no prompt cap.