Django Pandas to http response (download file)

Question

Python: 2.7.11

Django: 1.9

Pandas: 0.17.1

How should I go about creating a potentially large xlsx file download? I\'m creating a xlsx file with panda

Answer 1

Just wanted to share a class-based view approach to this, using elements from the answers above. Just override the get method of a Django View. My model has a JSON field which contains the results of dumping a dataframe to JSON with the to_json method.

Python version is 3.6 with Django 1.11.

# models.py
from django.db import models
from django.contrib.postgres.fields import JSONField

class myModel(models.Model):
    json_field = JSONField(verbose_name="JSON data")

# views.py
import pandas as pd
from io import BytesIO as IO
from django.http import HttpResponse
from django.views import View

from .models import myModel

class ExcelFileDownloadView(View):
    """
    Allows the user to download records in an Excel file
    """

    def get(self, request, *args, **kwargs):

        obj = myModel.objects.get(pk=self.kwargs['pk'])
        excel_file = IO()
        xlwriter = pd.ExcelWriter(excel_file, engine='xlsxwriter')
        pd.read_json(obj.json_field).to_excel(xlwriter, "Summary")
        xlwriter.save()
        xlwriter.close()

        excel_file.seek(0)

        response = HttpResponse(excel_file.read(),
                                content_type='application/vnd.openxmlformats-officedocument.spreadsheetml.sheet')

        response['Content-Disposition'] = 'attachment; filename="excel_file.xlsx"'
        return response

# urls.py
from django.conf.urls import url
from .views import ExcelFileDownloadView

urlpatterns = [
    url(r'^mymodel/(?P<pk>\d+)/download/$', ExcelFileDownloadView.as_view(), name="excel-download"),]

Answer 2

You're mixing two requirements that should be separate:

1. Creating a .xlsx file using python or pandas--it looks like you're good on this part.

2. Serving a downloadable file (django); see this post or maybe this one

Answer 3

Jmcnamara is pointing you in the rigth direction. Translated to your question you are looking for the following code:

sio = StringIO()
PandasDataFrame = pandas.DataFrame(self.csvdict)
PandasWriter = pandas.ExcelWriter(sio, engine='xlsxwriter')
PandasDataFrame.to_excel(PandasWriter, sheet_name=sheetname)
PandasWriter.save()

sio.seek(0)
workbook = sio.getvalue()

response = StreamingHttpResponse(workbook, content_type='application/vnd.openxmlformats-officedocument.spreadsheetml.sheet')
response['Content-Disposition'] = 'attachment; filename=%s' % filename

Notice the fact that you are saving the data to the StringIO variable and not to a file location. This way you prevent the file being saved before you generate the response.

Answer 4

Maybe a bit off-topic, but it's worth pointing out that the to_csv method is generally faster than to_excel, since excel contains format information of the sheets. If you only have data and not formatting information, consider to_csv. Microsoft Excel can view and edit csv files with no problem.

One gain by using to_csv is that to_csv function can take any file-like object as the first argument, not only a filename string. Since Django response object is file-like, to_csv function can directly write to it. Some codes in your view function will look like:

df = <your dataframe to be downloaded>
response = HttpResponse(content_type='text/csv')
response['Content-Disposition'] = 'attachment; filename=<default filename you wanted to give to the downloaded file>'
df.to_csv(response, index=False)
return response

Reference:

1. https://gist.github.com/jonperron/733c3ead188f72f0a8a6f39e3d89295d
2. https://docs.djangoproject.com/en/2.1/howto/outputting-csv/

Answer 5

I will elaborate on what @jmcnamara wrote. This if for the latest versions of Excel, Pandas and Django. The import statements would be at the top of your views.py and the remaining code could be in a view:

import pandas as pd
from django.http import HttpResponse
try:
    from io import BytesIO as IO # for modern python
except ImportError:
    from io import StringIO as IO # for legacy python

# this is my output data a list of lists
output = some_function()
df_output = pd.DataFrame(output)

# my "Excel" file, which is an in-memory output file (buffer) 
# for the new workbook
excel_file = IO()

xlwriter = pd.ExcelWriter(excel_file, engine='xlsxwriter')

df_output.to_excel(xlwriter, 'sheetname')

xlwriter.save()
xlwriter.close()

# important step, rewind the buffer or when it is read() you'll get nothing
# but an error message when you try to open your zero length file in Excel
excel_file.seek(0)

# set the mime type so that the browser knows what to do with the file
response = HttpResponse(excel_file.read(), content_type='application/vnd.openxmlformats-officedocument.spreadsheetml.sheet')

# set the file name in the Content-Disposition header
response['Content-Disposition'] = 'attachment; filename=myfile.xlsx'

return response

Answer 6

With Pandas 0.17+ you can use a StringIO/BytesIO object as a filehandle to pd.ExcelWriter. For example:

import pandas as pd
import StringIO

output = StringIO.StringIO()

# Use the StringIO object as the filehandle.
writer = pd.ExcelWriter(output, engine='xlsxwriter')

# Write the data frame to the StringIO object.
pd.DataFrame().to_excel(writer, sheet_name='Sheet1')
writer.save()
xlsx_data = output.getvalue()

print len(xlsx_data)

After that follow the XlsxWriter Python 2/3 HTTP examples.

For older versions of Pandas you can use this workaround.