Major Description
I think we should relax our checks on when to not trust the content-length header from Origin. We have a particularly troublesome case, which breaks youtube and possibly other streaming media sites:
////////////////////////////////////////////////////////
// this is the case that the origin server sent //
// us content length. Experience says that many //
// origin servers that do not support keep-alive //
// lie about the content length. to avoid truncation //
// of docuemnts from such server, if the server is //
// not keep-alive, we set to read until the server //
// closes the connection. We sent the maybe bogus //
// content length to the browser, and we will correct //
// the cache if it turnrd out that the servers lied. //
////////////////////////////////////////////////////////
if (incoming_response->presence(MIME_PRESENCE_CONTENT_LENGTH)) {
if (s->current.server->keep_alive == HTTP_NO_KEEPALIVE) {
s->hdr_info.trust_response_cl = false;
s->hdr_info.response_content_length = HTTP_UNDEFINED_CL
I'd like to suggest that we make this case optional (enabled / disabled via a records.config option). This would obviously require extensive testing, and probably should be enabled by default (i.e. preserver current behaviour).
From the BIS RFC:
http://tools.ietf.org/html/draft-ietf-httpbis-p1-messaging-12#section-3.3
If the actual number of
octets sent in the message is more than the indicated Content-
Length, the recipient MUST only process the message-body up to
the field value's number of octets; the remainder of the message
MUST either be discarded or treated as the next message in a
pipeline.