You score 50 million records each weekend. Which endpoint fits?
Why B is the answer
A batch endpoint
Batch endpoints process large datasets in jobs.
In detail
- A batch endpoint handles large volumes that can wait.
- Write results where downstream jobs can read them.
Use case
An illustrative scenario, not a real incident.
A bank scores 50 million records each weekend. A batch endpoint handles the volume, with no always-on capacity.
See it in different ways
flowchart TD
A{"Is a caller waiting?"} -->|Yes| B["Online endpoint"]
A -->|No| C["Batch endpoint"]
B --> D["Low-latency scoring"]
C --> E["Scores large datasets in jobs"]stateDiagram-v2 [*] --> WhenNeeded WhenNeeded --> Online: during a request WhenNeeded --> Batch: large volume, can wait Online --> Deployments: stable URL, several deployments Batch --> [*] Deployments --> [*]
flowchart LR S0["Input data"] --> S1["Batch endpoint"] --> S2["Compute scales up"] --> S3["Output files"]
Key point: Large offline scoring suits batch endpoints.
Why the other options are wrong
- A. An MLflow experiment with one run per record
- An experiment records runs, and does not score data.
- C. A real-time endpoint, called once per record
- Per-record calls are slow and costly.
- D. A notebook that is run by hand each weekend
- A manual run does not scale.