Throttling errors appear only during peak hours, while average usage is well under your limit. What do you check first?
Why D is the answer
The deployment's rate limits and quota against peak traffic
Limits apply to bursts as well as averages, so compare peak demand with deployment capacity before changing anything else.
In detail
- Limits apply at each moment, not on average.
- Check rate limits and quota against peak traffic.
- Plan capacity from peak traffic, not the daily average.
Use case
An illustrative scenario, not a real incident.
An app averages well under its limit, yet it is throttled at 9am. Peak demand passes the deployment's rate limit, so the team raises its TPM for peaks.
See it in different ways
flowchart LR A["Subscription quota per region, model and type"] --> B["Deployment A: TPM"] A --> C["Deployment B: TPM"] B --> D["Requests over the limit get throttled"]
stateDiagram-v2 [*] --> ModelQuota ModelQuota --> DeploymentA: assign TPM ModelQuota --> DeploymentB: assign TPM DeploymentA --> ModelQuota: reduce TPM to free capacity DeploymentB --> Throttled: peak above its limit
flowchart TD
Q{"When does throttling happen?"}
Q -->|"At peaks only"| A0["Check rate limits and quota"]
Q -->|"All the time"| A1["Quota is far too low"]Key point: Limits apply at each moment, not on average.
Why the other options are wrong
- A. The vector index dimensions and how often the index is refreshed
- Index settings do not throttle model calls.
- B. The content filter severity set on the deployment
- A filter blocks content, not request volume.
- C. The model's temperature and top-p settings for those requests
- Those settings change output, not throttling.