A web app must show model output as it is generated. Which integration is right?
Why A is the answer
Use the streaming response API and pass chunks to the client
Streaming returns tokens as they are generated, and the app forwards them to the client.
In detail
- The streaming API sends chunks as they are generated.
- Your app must forward chunks to the client too.
Use case
An illustrative scenario, not a real incident.
A chat app shows words as they arrive, so a 20-second answer feels quick. Waiting for the full response would feel broken.
See it in different ways
flowchart LR A["Browser"] --> B["API Gateway WebSocket or streaming API"] B --> C["Lambda"] C --> D["Bedrock streaming response"] D --> A
mindmap
root((Inference))
On demand
Variable traffic
Provisioned Throughput
Steady and high
Batch
Offline and cheaper
Cross-Region
Spreads peaks
Streaming
Show text as it arrivessequenceDiagram participant P0 as Browser participant P1 as App participant P2 as Model P0->>P1: Question P1->>P2: Streaming request P2-->>P1: Chunks P1-->>P0: Forward chunks
Key point: Stream for perceived speed.
Why the other options are wrong
- B. Run a nightly batch job and cache the output for each user
- A nightly job cannot answer live.
- C. Poll a DynamoDB table every few seconds for partial results
- Polling lags, and adds load.
- D. Send the full result over a WebSocket once generation finishes
- Sending at the end is not streaming.