What is the programming model for Dataflow pipelines?
Why B is the answer
Apache Beam
Dataflow runs Apache Beam pipelines.
In detail
- Dataflow pipelines are written with Apache Beam.
- The same model covers batch and streaming.
Use case
An illustrative scenario, not a real incident.
A team writes a pipeline in Apache Beam, and Dataflow runs it.
See it in different ways
flowchart TD
A{"What do you have?"} -->|"New batch or streaming pipeline"| B["Dataflow with Apache Beam"]
A -->|"Existing Spark or Hadoop"| C["Dataproc"]
A -->|"Visual, code-free integration"| D["Cloud Data Fusion"]
A -->|"SQL transformations in BigQuery"| E["Dataform"]mindmap
root((Processing))
Dataflow
Apache Beam
Dataproc
Existing Spark
Data Fusion
Visual, code free
Dataform
SQL in BigQuery
Managed Airflow
DAG orchestrationmindmap
root(("A Dataflow pipeline"))
n1["Programming model"]
n2["Apache Beam"]
n3["Managed runner"]
n4["Dataflow"]Key point: Dataflow runs Beam pipelines.
Why the other options are wrong
- A. A BigQuery reservation
- A reservation buys capacity.
- C. Spark SQL
- Spark SQL runs on Spark, not on Dataflow.
- D. Apache Hive
- Hive is not Dataflow's model.