A model update changes agent behavior in production. What limits the risk?
Why C is the answer
Pinned versions and staged rollout
Pinning prevents silent change, and stages limit impact.
In detail
- Pinning prevents silent change, and stages limit the impact.
- Silent updates are a real risk.
Use case
An illustrative scenario, not a real incident.
A provider updates a model, but a pinned version keeps behavior stable until staged tests pass.
See it in different ways
flowchart LR A["Agent run"] --> B["Trace of each step"] B --> C["Dashboards and alerts"] C --> D["Root cause"] D --> E["Add a test and fix"]
stateDiagram-v2 [*] --> Running Running --> Incident Incident --> ReadTrace ReadTrace --> Fix Fix --> AddEvalCase AddEvalCase --> StagedRollout StagedRollout --> Running
stateDiagram-v2 state "Pinned version" as s0 state "Staged rollout" as s1 state "New pinned version" as s2 [*] --> s0 s0 --> s1: new version tested s1 --> s2: metrics hold s2 --> [*]
Key point: Pin versions and stage rollouts.
Why the other options are wrong
- A. Removing old tests after each release
- Tests protect you.
- B. Updating all models on a Friday night
- Timing is not a control.
- D. Automatic upgrades to each new model
- That brings silent change.