[HDFS-11708] Positional read will fail if replicas moved to different DNs after stream is opened

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.7.3
• Fix Version/s: 2.9.0, 2.7.4, 3.0.0-alpha4, 2.8.2
• Component/s: hdfs-client
• Target Version/s: 2.7.4
• Hadoop Flags: Reviewed
• Assignee: Vinayakumar B
• Reporter: Vinayakumar B
• Created: 27/Apr/17 05:00
• Updated: 24/Apr/18 20:52
• Resolved: 07/Jun/17 05:33

Description

Scenario:
1. File was written to DN1, DN2 with RF=2
2. File stream opened to read and kept. Block Locations are [DN1,DN2]
3. One of the replica (DN2) moved to another datanode (DN3) due to datanode dead/balancing/etc.
4. Latest block locations in NameNode will be DN1 and DN3 in the 'same order'
5. DN1 went down, but not yet detected as dead in NameNode.
6. Client start reading using positional read api "read(pos, buf[], offset, length)"

Issue Links

• depends upon: HDFS-11898 DFSClient#isHedgedReadsEnabled() should be per client flag
• is depended upon by: HDFS-11738 Hedged pread takes more time when block moved from initial locations