Using Django's collectstatic with boto S3 throws “Error 32: Broken Pipe” after a while

Question

I'm using boto with S3 to store my Django site's static files. When using the collectstatic command, it uploads a good chunk of the files perfectly before stopping at a file and throwing "Error 32: Broken Pipe." When I try to run the command again, it skips over the files it has already uploaded and starts at the file where it left off, before throwing the same error without having uploaded anything new.

Answer 1

The key seems to be to specify which AWS Endpoint your bucket is located in. I tried doing this a bunch of different ways, but the solution that finally worked for me was to create a config file for boto as specified in the documentation.

Here are the contents of the config file I created at ~/.boto:

[Credentials]
aws_access_key_id = XXXXXXXXXXXXXXXXXXXX
aws_secret_access_key = XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
[s3]
host=s3-us-west-2.amazonaws.com

Answer 2

BenWurth's answer is fine, but if you're like me and don't want to add a config file, you can alternatively use django-storages better maintained cousin "django-storages-redux" and just add an extra env var to settings.py

pip uninstall django-storages (if you have this installed)
pip install django-storages-redux

and then in settings.py

AWS_S3_HOST = 'YOUR-AWS-ENDPOINT' #in my case 's3-us-west-2.amazonaws.com'

Then everything should be good.

Answer 3

Old question but to fix this easily i just added the Environment variable "AWS_DEFAULT_REGION" with the region i was using (eg "ap-southeast-2"). This work locally (windows) and in AWS EB

Answer 4

I also had the problem only with jquery.js, probably because it is too big like @Kyle Falconer mentions. It had nothing to do with region in my case. I "solved" it by copying the file locally to the S3 bucket where it needed to be.