[HDFS-1907] BlockMissingException upon concurrent read and write: reader was doing file position read while writer is doing write without hflush

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.23.0
• Fix Version/s: 0.23.0
• Component/s: hdfs-client

Environment

Run on a real cluster. Using the latest 0.23 build.

• Hadoop Flags: Reviewed
• Assignee: John George
• Reporter: CW Chung
• Created: 10/May/11 01:34
• Updated: 15/Nov/11 00:52
• Resolved: 03/Jun/11 18:06

Description

BlockMissingException is thrown under this test scenario:
Two different processes doing concurrent file r/w: one read and the other write on the same file

• writer keep doing file write
• reader doing position file read from beginning of the file to the visible end of file, repeatedly

The reader is basically doing:
byteRead = in.read(currentPosition, buffer, 0, byteToReadThisRound);
where CurrentPostion=0, buffer is a byte array buffer, byteToReadThisRound = 1024*10000;

Usually it does not fail right away. I have to read, close file, re-open the same file a few times to create the problem. I'll pose a test program to repro this problem after I've cleaned up a bit my current test program.

Issue Links

• is related to: HDFS-2029 Improve TestWriteRead