How do I disable 'Transfer-Encoding: chunked' encoding in Varnish?

Question

Using Varnish 4, I have a set of backends that\'re responding with a valid Content-Length header and no Transfer-Encoding header.

On the first

Answer 1

For the time being, do_stream = false will do what you want.

Avoiding chunked encoding for the case where the backend sends unchunked is a possible future improvement to Varnish.

Example:

sub vcl_backend_response {
        if(beresp.http.Content-Type ~ "video") {
                set beresp.do_stream = false;
                set beresp.do_gzip = false;
                //set resp.http.Content-Length = beresp.http.Content-Length;
        }
        if(beresp.http.Edge-Control == "no-store") {
                set beresp.uncacheable = true;
                set beresp.ttl = 60s;
                set beresp.http.Smug-Cacheable = "No";
                return(deliver);
        }
}

Answer 2

Upgraded from varnish 4.0 to 5.2 and now this works correctly also for the 1st request.

Answer 3

So the solution is not at all intuitive, but you must enable esi processing:

sub vcl_backend_response {
        set beresp.do_esi = true;

        if(beresp.http.Content-Type ~ "video") {
                set beresp.do_stream = true;
                set beresp.do_gzip = false;
                //set resp.http.Content-Length = beresp.http.Content-Length;
        }
        if(beresp.http.Edge-Control == "no-store") {
                set beresp.uncacheable = true;
                set beresp.ttl = 60s;
                set beresp.http.Smug-Cacheable = "No";
                return(deliver);
        }
}

So I discovered this by browsing the source code.

In particular, Varnish does this:

if (!req->disable_esi && req->obj->esidata != NULL) {
    /* In ESI mode, we can't know the aggregate length */
    req->res_mode &= ~RES_LEN;
    req->res_mode |= RES_ESI;
}

The above code sets the res_mode flag.

A little while later:

if (!(req->res_mode & (RES_LEN|RES_CHUNKED|RES_EOF))) {
    /* We havn't chosen yet, do so */
    if (!req->wantbody) {
        /* Nothing */
    } else if (req->http->protover >= 11) {
        req->res_mode |= RES_CHUNKED;
    } else {
        req->res_mode |= RES_EOF;
        req->doclose = SC_TX_EOF;
    }
}

This sets the res_mode flag to RES_CHUNKED if the HTTP protocol is HTTP/1.1 or higher (which it is in your example) and the res_mode flag isn't set. Now even later:

if (req->res_mode & RES_CHUNKED)
    http_SetHeader(req->resp, "Transfer-Encoding: chunked");

Varnish sends the chuncked transfer encoding if the RES_CHUNKED flag is set.

The only way I see to effectively disable this is by enabling ESI mode. It gets disabled in a few other ways, but those aren't practical (e.g. for HTTP HEAD requests or pages with a 304 status code).