Amazon redshift: bulk insert vs COPYing from s3

Question

I have a redshift cluster that I use for some analytics application. I have incoming data that I would like to add to a clicks table. Let\'s say I have ~10 new \'cl

Answer 1

S3 copy works faster in case of larger data loads. when you have say thousands-millions of records needs to be loaded to redshift then s3 upload + copy will work faster than insert queries.

S3 copy works in parallel mode.

When you create table and do insert then there is limit for batch size. The maximum size for a single SQL is 16 MB. So you need to take care size of SQL Batch ( depends on size of each insert query)

The S3 copy automatically applies encoding ( compression) for your table. When your create table and do sample load using copy then you can see compression automatically applied.

But if you are using insert command for beginning you will notice no compression applied which will result more space for table in redshift and slow query process timing in some cases.

If you wish to use insert commands, then create table with each column has applied encodings to save space and faster response time.