How to load data from S3 to PostgreSQL RDS

Question

I have a need to load data from S3 to Postgres RDS (around 50-100 GB) I don\'t have the option to use AWS Data Pipeline and I am looking for something similar to using the C

Answer 1

http://docs.aws.amazon.com/redshift/latest/dg/t_loading-tables-from-s3.html

Use the COPY command to load a table in parallel from data files on Amazon S3. You can specify the files to be loaded by using an Amazon S3 object prefix or by using a manifest file.

The syntax to specify the files to be loaded by using a prefix is as follows:

copy  from 's3:///'
authorization;

update

Another option is to mount s3 and use direct path to the csv with COPY command. I'm not sure If it will hold 100GB effectively, but worth of trying. Here is some list of options on software.

Yet another option would be "parsing" s3 file part by part with something described here to a file and COPY from named pipe, described here

And the most obvious option to just download file to local storage and use COPY I don't cover at all

Also worth of mentioning would be s3_fdw (status unstable). Readme is very laconic, but I assume you could create a foreign table leading to s3 file. Which itself means you can load data to other relation...