Amazon redshift: bulk insert vs COPYing from s3

Question

I have a redshift cluster that I use for some analytics application. I have incoming data that I would like to add to a clicks table. Let\'s say I have ~10 new \'cl

Answer 1

My test results differ a bit. I was loading CSV file to Redshift from OS Windows desktop.

• Row insert was the slowest.
• Multi-row insert was 5 times faster than row inset.
• S3+COPY was 3 times faster than multi-row insert.

What contributed to faster bulk S3+COPY insert.

• The fact that you do not have to parse insert statement from CSV line.
• Stream was compressed before multipart upload to S3.
• COPY command was extremely fast.

I compiled all my findings into one Python script CSV_Loader_For_Redshift