What is the purpose of A/B testing two models?
Why D is the answer
To compare versions on real outcomes
A/B testing compares two versions on real outcomes with real users.
In detail
- A/B tests compare versions on real outcomes.
- Run it long enough for a clear result.
Use case
An illustrative scenario, not a real incident.
Two models each serve some users, and the team compares purchases between them.
See it in different ways
flowchart LR
A["Traffic"] -->|"95%"| B["Current model"]
A -->|"5%"| C["New model"]
C --> D{"Metrics better?"}
D -->|Yes| E["Increase traffic"]
D -->|No| F["Roll back"]sequenceDiagram participant T as Traffic participant C as Current model participant N as New model participant M as Metrics T->>C: Most requests T->>N: Small share C->>M: Outcomes N->>M: Outcomes M-->>T: Shift or roll back
sequenceDiagram participant P0 as Router participant P1 as Model A participant P2 as Model B P0->>P1: Half of users P0->>P2: Half of users
Key point: A/B tests compare real outcomes.
Why the other options are wrong
- A. To reduce the size of a model
- It has no effect on size.
- B. To encrypt traffic between services
- It does not encrypt.
- C. To replace all model evaluation
- It adds to evaluation.