You need to load large files from S3 into Amazon Redshift quickly. What is the recommended way?
Why D is the answer
The COPY command
COPY loads in parallel and is built for bulk ingestion. Row-by-row inserts are very slow.
In detail
- COPY loads large files from S3 in parallel.
- Split files to help parallel loading.
Use case
An illustrative scenario, not a real incident.
A nightly load of large S3 files uses COPY, so Redshift reads them in parallel.
See it in different ways
mindmap
root((Distribution))
ALL
Small dimensions
KEY
Large joined tables
Care
VACUUM
ANALYZEflowchart TD
Q{"How much data?"}
Q -->|"Bulk files"| A0["COPY"]
Q -->|"A few rows"| A1["INSERT"]Key point: COPY is the bulk load path.
Why the other options are wrong
- A. Lambda calling SELECT in a loop
- SELECT reads data.
- B. Athena CTAS into S3
- That writes to S3, not Redshift.
- C. Many single-row INSERT statements
- Single-row inserts are very slow.