You need offline summaries of millions of records. What is the cost-effective option?
Why B is the answer
Batch inference
Batch inference suits large, asynchronous workloads.
In detail
- Batch inference suits offline work, at a lower cost.
- Results come back later.
Use case
An illustrative scenario, not a real incident.
A company summarizes five years of reviews overnight, with no user waiting, using batch inference.
See it in different ways
flowchart TD
A{"What do you need?"} -->|"Variable traffic, managed models"| B["Bedrock on-demand"]
A -->|"Consistent high throughput"| C["Bedrock Provisioned Throughput"]
A -->|"Your own model container"| D["SageMaker AI endpoint"]
A -->|"Offline volume"| E["Batch inference"]mindmap
root((Inference))
On demand
Variable traffic
Provisioned Throughput
Steady and high
Batch
Offline and cheaper
Cross-Region
Spreads peaks
Streaming
Show text as it arrivesflowchart LR S0["Upload input"] --> S1["Batch job"] --> S2["Collect output"]
Key point: Batch suits large asynchronous work.
Why the other options are wrong
- A. A provisioned model that is kept running all the time for the job
- Keeping capacity running is wasteful.
- C. A real-time call for each record, sent in one tight loop
- A tight loop is costly and slow.
- D. A larger on-demand quota, with one synchronous call per record
- Synchronous calls cost more.