[HDFS-3701] HDFS may miss the final block when reading a file opened for writing if one of the datanode is dead

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.0.3
• Fix Version/s: 1.1.0
• Component/s: hdfs-client
• Hadoop Flags: Reviewed
• Assignee: Nicolas Liochon
• Reporter: Nicolas Liochon
• Created: 23/Jul/12 10:50
• Updated: 12/Mar/13 00:35
• Resolved: 25/Sep/12 01:50

Description

When the file is opened for writing, the DFSClient calls one of the datanode owning the last block to get its size. If this datanode is dead, the socket exception is shallowed and the size of this last block is equals to zero. This seems to be fixed on trunk, but I didn't find a related Jira. On 1.0.3, it's not fixed. It's on the same area as HDFS-1950 or HDFS-3222.

Issue Links

• duplicates: HDFS-3965 DFSInputStream should not eat up exceptions if file is under construction
• is related to: HDFS-3222 DFSInputStream#openInfo should not silently get the length as 0 when locations length is zero for last partial block.
• is related to: HBASE-6751 Too many retries, leading a a delay to read the HLog after a datanode failure
• relates to: HBASE-6401 HBase may lose edits after a crash if used with HDFS 1.0.3 or older
• relates to: HDFS-4590 Add more Unit Test Case for HDFS-3701 HDFS Miss Final Block Reading when File is Open for Write