A job takes 30 minutes per request on large payloads, and clients can pick up results later. Which option fits?
Why B is the answer
Asynchronous inference
Asynchronous inference queues long-running requests with large payloads and returns results later.
In detail
- Asynchronous inference handles long runs and large payloads.
- Clients collect results later.
- Results are written to S3.
Use case
An illustrative scenario, not a real incident.
Video analysis takes 30 minutes per file. Asynchronous inference queues each request, and clients collect results later.
See it in different ways
flowchart TD
A{"Traffic pattern"} -->|"Steady, low latency"| B["Real-time endpoint"]
A -->|"Sporadic, idle gaps"| C["Serverless inference"]
A -->|"Long jobs, large payloads"| D["Asynchronous inference"]
A -->|"A whole dataset offline"| E["Batch transform"]stateDiagram-v2 [*] --> Traffic Traffic --> RealTime: steady and low latency Traffic --> Serverless: sporadic, cold starts are fine Traffic --> Asynchronous: large payloads, long runs Traffic --> BatchTransform: a whole dataset
sequenceDiagram participant P0 as Client participant P1 as Async endpoint participant P2 as S3 P0->>P1: Submit request P1-->>P0: Accepted P1->>P2: Write result P0->>P2: Collect later
Key point: Long jobs with large payloads suit asynchronous inference.
Why the other options are wrong
- A. A real-time endpoint
- Real-time clients would wait too long.
- C. An EC2 cron job
- A cron job is not a managed service for this.
- D. Serverless inference
- Serverless has shorter limits on run time.