Traffic is sporadic with long idle periods, and a few seconds of cold start is acceptable. Which option is most cost-effective?
Why D is the answer
SageMaker Serverless Inference
Serverless inference scales to zero when idle. Always-on endpoints charge you through the quiet periods.
In detail
- Serverless Inference suits idle periods, if a cold start is acceptable.
- Expect cold starts after idle periods.
Use case
An illustrative scenario, not a real incident.
An internal tool gets a few requests a day. Serverless Inference scales to zero between them, so idle time costs nothing.
See it in different ways
flowchart TD
A{"Traffic pattern"} -->|"Steady, low latency"| B["Real-time endpoint"]
A -->|"Sporadic, idle gaps"| C["Serverless inference"]
A -->|"Long jobs, large payloads"| D["Asynchronous inference"]
A -->|"A whole dataset offline"| E["Batch transform"]stateDiagram-v2 [*] --> Traffic Traffic --> RealTime: steady and low latency Traffic --> Serverless: sporadic, cold starts are fine Traffic --> Asynchronous: large payloads, long runs Traffic --> BatchTransform: a whole dataset
flowchart TD
Q{"How does traffic behave?"}
Q -->|"Sporadic, cold start is fine"| A0["Serverless inference"]
Q -->|"Steady, low latency"| A1["Real-time endpoint"]Key point: Sporadic traffic suits serverless inference.
Why the other options are wrong
- A. A batch transform job each minute
- A job each minute is the wrong tool.
- B. A dedicated GPU cluster
- A dedicated cluster sits idle.
- C. A real-time endpoint on large instances
- You would pay while it sits idle.