You need predictions for a full table overnight. Which inference type fits?
Why A is the answer
Batch inference
Batch suits large, offline work.
In detail
- Batch inference suits a full table overnight.
- Choose online inference when someone is waiting.
Use case
An illustrative scenario, not a real incident.
A team scores a whole table overnight in a batch job, and no endpoint stays running.
See it in different ways
flowchart TD
A{"Is a caller waiting?"} -->|No| B["Batch inference"]
A -->|Yes| C{"How much control?"}
C -->|"Managed"| D["Agent Platform endpoint"]
C -->|"Own containers"| E["Cloud Run or GKE"]mindmap
root((Serving))
Online
Autoscaled backend
Batch
Overnight tables
Private endpoint
Internal only
Edge
No connection
Custom container
Any frameworkflowchart TD
Q{"When is the answer needed?"}
Q -->|"Later"| A0["Batch inference"]
Q -->|"Right now"| A1["Online inference"]Key point: Batch suits offline work.
Why the other options are wrong
- B. Online inference for each row
- One call per row is slow and costly.
- C. A canary rollout
- A canary tests a model.
- D. An A/B test
- An A/B test compares models.