问题
Using Varnish 4, I have a set of backends that're responding with a valid Content-Length
header and no Transfer-Encoding
header.
On the first hit from a client, rather than responding to the client with those headers, Varnish is dropping the Content-Length
header and adding Transfer-Encoding: chunked
to the response. (Interestingly, the payload doesn't appear to have any chunks in it - it's one contiguous payload).
This causes serious problems for clients like Flash video players that are trying to do segment-size, bandwidth, etc analysis based on the Content-Length
header. Their analysis fails, and they can't do things like multi-bitrate streaming, etc.
I've tried a number of semi-obvious things like:
beresp.do_stream = true
beresp.do_gzip = false
unset req.http.Accept-Encoding
Sample backend response:
HTTP/1.1 200 OK
Cache-Control: public, max-age=600
Content-Type: video/mp4
Date: Tue, 13 May 2014 19:44:35 GMT
Server: Apache
Content-Length: 796618
Connection: keep-alive
Sample varnish response:
HTTP/1.1 200 OK
Server: Apache
Cache-Control: public, max-age=600
Content-Type: video/mp4
Date: Tue, 13 May 2014 23:10:06 GMT
X-Varnish: 2
Age: 0
Transfer-Encoding: chunked
Accept-Ranges: bytes
Subsequent loads of the object do including the Content-Length
header, just not the first load into cache.
VCL: https://gist.github.com/onethumb/e64a405cc579909cace1
varnishlog output: https://gist.github.com/onethumb/e66a2bc4727a3a5340b6
Varnish Trac: https://www.varnish-cache.org/trac/ticket/1506
回答1:
For the time being, do_stream = false
will do what you want.
Avoiding chunked encoding for the case where the backend sends unchunked is a possible future improvement to Varnish.
Example:
sub vcl_backend_response {
if(beresp.http.Content-Type ~ "video") {
set beresp.do_stream = false;
set beresp.do_gzip = false;
//set resp.http.Content-Length = beresp.http.Content-Length;
}
if(beresp.http.Edge-Control == "no-store") {
set beresp.uncacheable = true;
set beresp.ttl = 60s;
set beresp.http.Smug-Cacheable = "No";
return(deliver);
}
}
回答2:
So the solution is not at all intuitive, but you must enable esi processing:
sub vcl_backend_response {
set beresp.do_esi = true;
if(beresp.http.Content-Type ~ "video") {
set beresp.do_stream = true;
set beresp.do_gzip = false;
//set resp.http.Content-Length = beresp.http.Content-Length;
}
if(beresp.http.Edge-Control == "no-store") {
set beresp.uncacheable = true;
set beresp.ttl = 60s;
set beresp.http.Smug-Cacheable = "No";
return(deliver);
}
}
So I discovered this by browsing the source code.
In particular, Varnish does this:
if (!req->disable_esi && req->obj->esidata != NULL) {
/* In ESI mode, we can't know the aggregate length */
req->res_mode &= ~RES_LEN;
req->res_mode |= RES_ESI;
}
The above code sets the res_mode
flag.
A little while later:
if (!(req->res_mode & (RES_LEN|RES_CHUNKED|RES_EOF))) {
/* We havn't chosen yet, do so */
if (!req->wantbody) {
/* Nothing */
} else if (req->http->protover >= 11) {
req->res_mode |= RES_CHUNKED;
} else {
req->res_mode |= RES_EOF;
req->doclose = SC_TX_EOF;
}
}
This sets the res_mode
flag to RES_CHUNKED
if the HTTP protocol is HTTP/1.1
or higher (which it is in your example) and the res_mode flag isn't set. Now even later:
if (req->res_mode & RES_CHUNKED)
http_SetHeader(req->resp, "Transfer-Encoding: chunked");
Varnish sends the chuncked transfer encoding if the RES_CHUNKED
flag is set.
The only way I see to effectively disable this is by enabling ESI mode. It gets disabled in a few other ways, but those aren't practical (e.g. for HTTP HEAD requests or pages with a 304 status code).
回答3:
Upgraded from varnish 4.0 to 5.2 and now this works correctly also for the 1st request.
来源:https://stackoverflow.com/questions/23643233/how-do-i-disable-transfer-encoding-chunked-encoding-in-varnish