Best way to upload large csv files using python flask

Question

Requirement: To Upload files using flask framework. Once uploaded to the server user should be able to see the file in UI.

Current code

Answer 1

On the flask side make sure you have the MAX_CONTENT_LENGTH config value set high enough:

app.config['MAX_CONTENT_LENGTH'] = 100 * 1024 * 1024  # 100MB limit

Also you may want to look into the Flask-Upload extension.

There is another SO post similar to this one: Large file upload in Flask.

Other than that you problem may be a timeouts somewhere along the line. What does the rest of your stack look like? Apache? Nginx and Gunicorn? Are you getting a Connection reset error, Connection timed out error or does it just hang?

If you are using Nginx try setting proxy_read_timeout to a value high enough for the upload to finish. Apache may also have a default setting causing you trouble if that is what you are using. It's hard to tell without knowing more about your stack and what the error is that you are getting and what the logs are showing.