Gzipping all HTTP traffic with Pyramid

Question

I am creating a mobile service based on Pyramid framework. Because it\'s mobile everything to reduce bandwidth usage is plus. I am considering gzipping all the traffic, even

Answer 1

First of all I should stress that you should do this on the web server level (nginx or apache). There are several reasons for this:

1. Performance - If you do this in Python you are using one of your threads that could be handling requests to do cpu-intensive compression. This is way less efficient than allowing your optimized web server to handle it.

2. Blocking - Most GZip middleware will block your responses, buffering the body so that it can compress the entire response. This is a huge problem if you are attempting to stream any response back to the client because it will get caught in middleware. This is actually a violation of PEP333, the WSGI specification.

With all of this in mind, it might make sense to do it in Python at least for debugging purposes during development.

Since you're already using Pyramid then you have Paste installed. Thus you can simply add the paste.gzipper.GzipMiddleware to your application's pipeline like so:

[filter:gzip]
use = egg:Paste#gzip
compress_level = 6

[pipeline:main]
pipeline =
    gzip
    app

Obviously if you don't want to change the compression level from the default of 6 you can simply add the egg:Paste#gzip to the pipeline in place of configuring the filter and giving it a custom name (gzip).

Answer 2

You can still get per-request compression statistics with Apache. I created a deflate.log like so:

DeflateFilterNote Input instream
DeflateFilterNote Output outstream
DeflateFilterNote Ratio ratio
LogFormat '%a %v %{outstream}n/%{instream}n (%{ratio}n%%) "%r"' deflate
CustomLog /var/log/httpd/deflate.log deflate

Now I get log entries like:

ip domain -/- (-%) "GET /wp-content/uploads/2010/03/favicon.ico HTTP/1.1"
ip domain 10995/52406 (20%) "GET /2006/07/19/ HTTP/1.0"
ip domain 1873/7891 (23%) "POST /registration/regForm HTTP/1.1"

that I can analyze to my heart's content.

Answer 3

Here are a couple of options at the WSGI level:

• GZip Compression Egress Filter
• ibkon-wsgi-gzip-middleware