A team needs custom Spark tuning on very large clusters. Which service fits?
Why B is the answer
Amazon EMR
EMR gives control over clusters and frameworks.
In detail
- EMR gives custom tuning on large clusters.
- More control means more to operate.
Use case
An illustrative scenario, not a real incident.
A team tunes Spark settings across a very large cluster, which EMR allows.
See it in different ways
flowchart TD
A{"Transform job"} -->|"Small, event-driven"| B["AWS Lambda"]
A -->|"Serverless Spark ETL"| C["AWS Glue"]
A -->|"Big clusters you tune"| D["Amazon EMR"]mindmap
root((Transform))
Glue
Serverless Spark
Job bookmarks
EMR
Custom tuning
Lambda
Small files
DataBrew
No code
Redshift
Stored proceduresmindmap
root(("Amazon EMR"))
n1["Control"]
n2["Cluster size"]
n3["Framework settings"]
n4["Cost"]
n5["More to operate"]Key point: EMR gives control over clusters.
Why the other options are wrong
- A. Amazon Quick Sight, with a calculated field for each step
- Quick Sight is for dashboards.
- C. Amazon Data Firehose, with a transformation Lambda function
- Firehose delivers data.
- D. AWS Glue DataBrew, with a visual recipe for each file
- DataBrew is visual.