[HDFS-4594] WebHDFS open sets Content-Length header to what is specified by length parameter rather than how much data is actually returned.

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.0.3-alpha
• Fix Version/s: 2.1.1-beta
• Component/s: webhdfs
• Target Version/s: 2.1.1-beta
• Hadoop Flags: Reviewed
• Assignee: Chris Nauroth
• Reporter: Arpit Gupta
• Created: 12/Mar/13 20:58
• Updated: 03/Sep/14 23:49
• Resolved: 20/Aug/13 22:51

Description

This was noticed on 2.0.3 alpha

Lets say we have a file of length x

We make an webhdfs open call specifying length=x+1

The response of the call redirected to the datanode sets the content length header to value x+1 rather than x.

Now this causes an error when the client tries to read the data.

For the test i was using HttpResponse.getEntity().getContent()

This failed with message "Premature end of Content-Length delimited message body (expected: 71898; received: 71897"

This was not seen in hadoop 1 as we did not set the content length header.