What is the purpose of a regression test set for an agent?
Why D is the answer
To catch failures that were fixed before
Past failures become tests.
In detail
- Past failures become tests, so they do not return.
- Each past failure becomes a test.
Use case
An illustrative scenario, not a real incident.
A fixed bug returns months later, and a regression test catches it.
See it in different ways
flowchart LR A["Agent run"] --> B["Trace of each step"] B --> C["Dashboards and alerts"] C --> D["Root cause"] D --> E["Add a test and fix"]
stateDiagram-v2 [*] --> Running Running --> Incident Incident --> ReadTrace ReadTrace --> Fix Fix --> AddEvalCase AddEvalCase --> StagedRollout StagedRollout --> Running
stateDiagram-v2 state "Failure found" as s0 state "Fixed" as s1 state "Test added" as s2 state "Runs on each change" as s3 [*] --> s0 s0 --> s1 s1 --> s2 s2 --> s3 s3 --> [*]
Key point: Regression tests stop repeats.
Why the other options are wrong
- A. To train the model's weights once again
- It does not retrain.
- B. To increase its peak request throughput
- It does not add throughput.
- C. To remove the need for production logs
- Logs are still needed.