A prompt change caused a production regression, and nobody can tell what changed. What practice prevents this?
Why C is the answer
Versioning prompts in Git with review and evaluation
Prompts are production artifacts. Git gives you history and rollback, and evaluation catches regressions before release.
In detail
- Git shows exactly what changed.
- Review and evaluation catch problems before release.
- Run evaluations automatically on prompt changes.
Use case
An illustrative scenario, not a real incident.
A prompt edit causes a regression, with no record of the change. With prompts in Git, review and evaluation run on each change, and a revert is simple.
See it in different ways
flowchart LR
A["Prompt in Git"] --> B["Variants"]
B --> C["Evaluate on a test set"]
C --> D{"Better?"}
D -->|Yes| E["Deploy"]
D -->|No| Amindmap
root((GenAIOps))
Prompts
In Git
Variants
Models
Fit on your data
Gradual rollout
Capacity
Standard
Provisioned
Evaluation
In CI/CD
On production tracesflowchart LR S0["Edit in Git"] --> S1["Pull request"] --> S2["Evaluation"] --> S3["Merge"]
Key point: Treat prompts like code.
Why the other options are wrong
- A. Sharing prompt changes over chat so the whole team sees them
- Chat messages are not a version history.
- B. Keeping the current prompt in code comments next to each call
- Comments do not control what runs.
- D. Editing prompts directly in the portal with a change note
- A portal change note is not a tested version.