What is the benefit of pipeline steps that can be rerun safely?
Why B is the answer
Failures can be retried without corrupting data
Idempotent steps make recovery simple.
In detail
- Safe reruns let failures be retried without corrupting data.
- MERGE or overwrite patterns make this possible.
Use case
An illustrative scenario, not a real incident.
A load fails halfway and is rerun with no duplicate rows, because the step is safe to repeat.
See it in different ways
stateDiagram-v2 [*] --> Load Load --> Validate: rules and row counts Validate --> Publish: passes Validate --> Quarantine: fails Quarantine --> Load: fix, then rerun safely Publish --> [*]
flowchart TD
Q{"Is the step idempotent?"}
Q -->|"Yes"| A0["Rerun safely"]
Q -->|"No"| A1["Risk duplicates"]Key point: Idempotent steps make recovery simple.
Why the other options are wrong
- A. Each step uses less memory
- It does not change memory use.
- C. Pipelines no longer need any monitoring
- You still monitor.
- D. Data no longer needs to be validated
- You still validate.