A multi-step pipeline has branching and retries, and you want a managed, visual workflow. What do you use?
Why D is the answer
AWS Step Functions
Step Functions orchestrates steps with branching and retries as a managed service.
In detail
- Step Functions gives a managed, visual workflow with branching and retries.
- Execution history helps with debugging.
Use case
An illustrative scenario, not a real incident.
A pipeline validates, branches on the result, and retries a failed load, all in one visual Step Functions workflow.
See it in different ways
flowchart LR
A["S3 event or schedule"] --> B["Step Functions or MWAA"]
B --> C["Glue job"]
C --> D{"Quality passed?"}
D -->|Yes| E["Load Redshift"]
D -->|No| F["Notify the team"]sequenceDiagram participant S as S3 participant E as EventBridge participant F as Step Functions participant N as SNS S->>E: Object created E->>F: Start the workflow F->>F: Glue job, with retries F-->>N: Alert on failure
flowchart TD
Q{"Did validation pass?"}
Q -->|"Yes"| A0["Load"]
Q -->|"No"| A1["Send to review"]Key point: Step Functions orchestrates with branches and retries.
Why the other options are wrong
- A. Amazon S3 Select
- S3 Select filters one object.
- B. Amazon EventBridge Scheduler
- That only starts things on a schedule.
- C. AWS Glue crawlers
- Crawlers read metadata.