Http Client does not fix incorrect content-lenght headers

• Key: HTTPCLIENT-600
• Type: Improvement
• Status: Closed
• Resolution: Invalid
• Priority: Major
• Assignee: Unassigned
• Reporter: Denis Valdenaire
• Component/s: HttpClient
• Affects Version/s: 3.1 Alpha 1

Description

I discovered that the method

addContentLengthRequestHeader (found in file methods/MultipartPostMethod.java) doesn't "fix" the content-lenght when this one is incorrect. It adds one if getRequestHeader("Content-Lenght") is null, but it should also verify that the content-lenght is correct.

I suggest something like :

long len = getRequestContentLength();
if (getRequestHeader("Content-Length") == null || getRequestHeader("Content-Length") != len) {
            setRequestHeader("Content-Length", String.valueOf(len));
}

Sending an incorrect Content-Length blocks the server if the string sent is smaller than announced : waiting for more, and finally reset the connection. If it's too big, you lose data.

I've seen this problem in a reverse proxy program (with httpclient communicating with the real servers) when the client send urlencoded data and this data is modified (partly urldecoded) but not the content-lenght.

Comments

[Roland Weber]

Hi Denis,

HttpClient behaves exactly as intended. If there is no Content-Length header, HttpClient tries to compute one.
If you think you know better what the Content-Length header should be, and set it explicitly, then you do so on
your own risk and responsibility. HttpClient will not modify headers that were set explicitly. That's mainly because
somebody might have to create requests that are actually invalid, but are similar to what some other, broken
HTTP application generates. If you want HttpClient to provide the content length, then just don't set it. If you
want to verify the value you set, then use RequestEntity.getContentLength() in your application.

Reverse proxies are supposed to know which headers can be sent on and which can not. A proxy that
modifies the request entity is also responsible for updating all entity headers that might be affected by
that change. Please open an issue against the reverse proxy program that misbehaves.

I suggest to mark this issue invalid.

cheers,
  Roland

[Oleg Kalnichevski]

I concur.

Oleg

[Denis Valdenaire]

Thanks for you answer.

I had this idea because i saw it in a perl module (lwp). I tried to send incorrect content-lenght to emulate the behaviour of a broken client but the perl code corrected it. So i had to modify the lib so it doesn't modify it.

At least can we just issue a warning and not change it ?

I agree that the content-length should not be modified if set explicitly but if false, it will cause an error that won't be easy to find. The warning (of debug, or trace) could help and will not modify the behaviour of the class.

But again, you're right, the program should not set - or forward the content-length if unsure.

Denis

[Roland Weber]

Hello Denis,

we won't add a String-to-int conversion (with exception handling for invalid numbers) just to generate a warning.
Besides, we had other users complaining about warnings that were generated about something they wanted
to do on purpose. Again: if you want the content length to be verified, please implement that in your application.

cheers,
  Roland

[Denis Valdenaire]

Hello Roland,

At the very least I will try to document it somewhere as a possible cause for a socket reset at the server side. We had a very hard time to debug this and only tcpdump was our friend....

Thanks for your help.

Best regards,

Denis