Which format is best for analytic queries on S3?
Why D is the answer
Apache Parquet
Columnar formats let engines read only the columns needed.
In detail
- Parquet is columnar, so queries read less.
- CSV forces reading the whole file.
Use case
An illustrative scenario, not a real incident.
A query needs two columns of 80. With Parquet, the engine reads only those two.
See it in different ways
flowchart LR A["Raw CSV"] --> B["Convert to Parquet"] B --> C["Partition by date"] C --> D["Queries scan less data"]
stateDiagram-v2 [*] --> RawCsv RawCsv --> Parquet: columnar, compressed Parquet --> Partitioned: by date Partitioned --> Compacted: fewer small files Compacted --> [*]
mindmap
root(("File formats"))
n1["Columnar"]
n2["Parquet"]
n3["Row-based"]
n4["CSV"]
n5["JSON"]Key point: Columnar suits analytic queries.
Why the other options are wrong
- A. CSV with a header in each file
- CSV is row based.
- B. One large JSON document per day
- One large JSON file is slow to read.
- C. Plain text with no fixed delimiter
- Plain text has no structure.