I am building a django/s3 application with the following behaviour:
Don't make your files public. This will prevent non-authorised users from accessing the files.
Then in your Django app, you can generate urls with a querystring that allows an authorised user to access an S3 file for a limited time.
The amazon docs for querystring request authentication have more information.