A new model must be tested on live traffic without affecting users. What do you use?
Why A is the answer
A shadow test
Shadow tests send a copy of traffic to the new model, and users see only the current model.
In detail
- A shadow test uses live traffic, and users see only the current model.
- Compare latency and errors between the two.
Use case
An illustrative scenario, not a real incident.
A new ranking model gets a copy of live traffic, while users see only the current model.
See it in different ways
flowchart LR A["Production traffic"] --> B["Current model answers users"] A --> C["Shadow copy goes to the new model"] C --> D["Compare the results"]
sequenceDiagram participant R as Model Registry participant S as Shadow test participant C as Canary traffic participant P as Production R->>S: Approved version S->>C: Live traffic, with no user impact C->>P: Shift more traffic P-->>C: Roll back if alarms fire
sequenceDiagram participant P0 as User participant P1 as Current model participant P2 as Shadow model P0->>P1: Request P1-->>P0: Response P1->>P2: Copy of traffic
Key point: Shadow tests use live traffic safely.
Why the other options are wrong
- B. A direct replacement of the production model
- Direct replacement affects users.
- C. A test on the training data only
- Training data is not live traffic.
- D. A deployment during a quiet hour with no checks
- No checks means no safety.