Events can arrive up to ten minutes late, and a five-minute count must stay accurate. What do you need in your streaming logic?
Why A is the answer
A windowing strategy that allows for late-arriving events
Windows define how events are grouped over time, and they must tolerate lateness, or late events are dropped or miscounted.
In detail
- Late events must still count in their window.
- The window has to allow for them.
- A longer tolerance gives more accuracy, and later results.
Use case
An illustrative scenario, not a real incident.
Sensors on a delivery fleet lose signal in tunnels and send readings late. A window that tolerates ten minutes of lateness still counts them.
See it in different ways
flowchart TD
A["Event arrives"] --> B{"Within the watermark?"}
B -->|Yes| C["Added to its window"]
B -->|No| D["Dropped or handled apart"]flowchart LR A["Events"] --> B["Eventstream"] B --> C["Eventhouse"] B --> D["Activator alert"] A --> E["Spark structured streaming"]
flowchart TD
Q{"Is it within the lateness tolerance?"}
Q -->|"Yes"| A0["Counted in its window"]
Q -->|"No"| A1["Dropped or handled apart"]Key point: Windows must allow for late events.
Why the other options are wrong
- B. A larger Spark executor so events are processed sooner
- Faster processing cannot make events arrive sooner.
- C. Row-level security to hold back events from late sources
- Row-level security is about access.
- D. A warehouse index on the event timestamp column
- An index does not change when events arrive.