A copy of files needs little code, while a complex transformation needs Python. How do you choose?
Why A is the answer
Use a pipeline for the copy, and a notebook for the transformation
Pipelines orchestrate and move data. Notebooks suit code-heavy transformation.
In detail
- A pipeline suits a low-code copy.
- A notebook suits Python.
- A pipeline can orchestrate a notebook as one activity.
Use case
An illustrative scenario, not a real incident.
A nightly job copies files from a partner and then runs complex matching. A pipeline does the copy and calls a notebook for the matching.
See it in different ways
flowchart LR A["Schedule or event trigger"] --> B["Pipeline"] B --> C["Copy activity"] C --> D["Notebook activity"] D --> E["Refresh or notify"]
mindmap
root((Fabric tools))
Pipeline
Copy
Schedule
ForEach
Notebook
Python
Complex logic
Dataflow Gen2
Low code
Triggers
Schedule
Eventflowchart LR S0["Pipeline starts"] --> S1["Copy activity"] --> S2["Notebook activity"] --> S3["Finished"]
Key point: Pipelines move data, and notebooks transform it.
Why the other options are wrong
- B. Use a dataflow for everything, regardless of complexity
- Dataflows do not suit every complex case.
- C. Use a shortcut for the copy, and T-SQL for the Python logic
- A shortcut does not transform data, and T-SQL does not run Python.
- D. Use a notebook for the copy, and a pipeline for the Python logic
- That is the wrong way around.