Several independent applications must read the same stream with low latency and replay older records. What should you pick?
Why A is the answer
Amazon Kinesis Data Streams
Kinesis Data Streams supports multiple consumers and replay within its retention period.
In detail
- Kinesis Data Streams lets several consumers read and replay records.
- Replay works within the retention period.
Use case
An illustrative scenario, not a real incident.
A fraud service, a dashboard and an archive job all read one stream, and the dashboard can replay yesterday's records.
See it in different ways
flowchart TD
A{"What do you need?"} -->|"Deliver to S3 or Redshift, no consumer code"| B["Amazon Data Firehose"]
A -->|"Custom consumers and replay"| C["Kinesis Data Streams"]
A -->|"Kafka compatibility"| D["Amazon MSK"]sequenceDiagram participant P as Producer participant K as Kinesis Data Streams participant A as Consumer A participant B as Consumer B P->>K: Records K->>A: Reads from its own position K->>B: Reads from its own position A->>A: Processes idempotently
sequenceDiagram participant P0 as Producer participant P1 as Data stream participant P2 as Fraud service participant P3 as Dashboard P0->>P1: Records P1->>P2: Read P1->>P3: Read or replay
Key point: Kinesis Data Streams supports many consumers and replay.
Why the other options are wrong
- B. AWS Glue streaming crawlers
- Crawlers read metadata.
- C. Amazon Athena federated queries
- Athena queries data, and does not stream it.
- D. Amazon SQS standard queues
- An SQS message goes to one consumer.