A streaming job must recover from worker failures. What does the design need?
Why A is the answer
Fault tolerance and managed restarts
A streaming job needs fault tolerance and managed restarts to recover from worker failures.
In detail
- Fault tolerance and managed restarts handle worker failures.
- Design steps to be safe when repeated.
Use case
An illustrative scenario, not a real incident.
A worker dies mid-stream, and the managed service restarts its work with no lost progress.
See it in different ways
flowchart LR A["Primary zone or region"] --> B["Replica"] A -->|"Failure"| C["Failover to the replica"]
sequenceDiagram participant P as Producers participant S as Pub/Sub participant D as Dataflow participant B as BigQuery P->>S: Events S->>D: Delivered with event time D->>D: Windows and watermarks D->>B: Aggregates
stateDiagram-v2 state "Worker running" as s0 state "Worker failed" as s1 state "Work restarted" as s2 [*] --> s0 s0 --> s1: failure s1 --> s2: managed restart s2 --> [*]
Key point: Design for worker failure.
Why the other options are wrong
- B. A manual restart by an operator each time
- Manual restarts are slow.
- C. No checkpointing at all
- With no checkpoints, state is lost.
- D. A single worker for simplicity
- One worker is a single point of failure.