An endpoint is often idle, yet its instances run all day. What is a good response?
Why B is the answer
Serverless inference or autoscaling
Match capacity to demand, so you do not pay for idle time.
In detail
- Serverless inference or autoscaling cuts idle cost.
- Choose by how much latency you can accept.
Use case
An illustrative scenario, not a real incident.
An endpoint sits idle most of the day. Moving to serverless inference or adding autoscaling matches cost to demand.
See it in different ways
flowchart TD
A{"Traffic pattern"} -->|"Steady, low latency"| B["Real-time endpoint"]
A -->|"Sporadic, idle gaps"| C["Serverless inference"]
A -->|"Long jobs, large payloads"| D["Asynchronous inference"]
A -->|"A whole dataset offline"| E["Batch transform"]stateDiagram-v2 [*] --> Traffic Traffic --> RealTime: steady and low latency Traffic --> Serverless: sporadic, cold starts are fine Traffic --> Asynchronous: large payloads, long runs Traffic --> BatchTransform: a whole dataset
flowchart TD
Q{"What does the traffic look like?"}
Q -->|"Long idle gaps"| A0["Serverless inference"]
Q -->|"Variable but constant"| A1["Autoscaling"]Key point: Match capacity to demand.
Why the other options are wrong
- A. A longer data capture window
- Longer capture costs more.
- C. A larger instance type
- A larger instance costs more.
- D. A second always-on instance
- A second instance costs more.