Uploading files to s3 using s3cmd in parallel

Question

I\'ve got a whole heap of files on a server, and I want to upload these onto S3. The files are stored with a .data extension, but really they\'re just a bunch of jpegs,pngs,zips

Answer 1

you can just use s3cmd-modified which allows you to put/get/sync with multiple workers in parallel

$ git clone https://github.com/pcorliss/s3cmd-modification.git $ cd s3cmd-modification $ python setup.py install $ s3cmd --parallel --workers=4 sync /source/path s3://target/path