A customer-facing app has steady, high token volume and needs predictable latency. Which deployment option should you evaluate?
Why A is the answer
Provisioned throughput units
Provisioned throughput reserves capacity for sustained high volume, which gives steadier latency than shared serverless capacity.
In detail
- Provisioned throughput suits steady, high volume.
- It gives predictable latency.
- Estimate the units from your real traffic.
Use case
An illustrative scenario, not a real incident.
A customer assistant has steady, heavy traffic and tight latency needs. Provisioned throughput units give predictable performance.
See it in different ways
flowchart TD
A{"Traffic pattern"} -->|"Variable or low"| B["Serverless API or standard"]
A -->|"Steady and high"| C["Provisioned throughput"]mindmap
root((GenAIOps))
Prompts
In Git
Variants
Models
Fit on your data
Gradual rollout
Capacity
Standard
Provisioned
Evaluation
In CI/CD
On production tracesflowchart TD
Q{"What is traffic like?"}
Q -->|"Steady and high"| A0["Provisioned throughput"]
Q -->|"Low or spiky"| A1["Standard"]Key point: Steady volume suits provisioned throughput.
Why the other options are wrong
- B. A free trial deployment
- A trial is not for production.
- C. A notebook running on a developer machine
- A developer machine cannot serve customers.
- D. A manual batch job once a day
- A daily batch cannot serve live requests.