[HDFS-7686] Re-add rapid rescan of possibly corrupt block feature to the block scanner

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 2.7.0
• Fix Version/s: 2.7.0
• Target Version/s: 2.7.0
• Assignee: Colin McCabe
• Reporter: Rushabh Shah
• Created: 27/Jan/15 20:12
• Updated: 29/Sep/17 19:49
• Resolved: 13/Feb/15 22:50

Description

When doing a transferTo (aka sendfile operation) from the DataNode to a client, we may hit an I/O error from the disk. If we believe this is the case, we should be able to tell the block scanner to rescan that block soon. The feature was originally implemented in HDFS-7548 but was removed by HDFS-7430. We should re-add it.

Issue Links

• breaks: HDFS-10627 Volume Scanner marks a block as "suspect" even if the exception is network-related
• is broken by: HDFS-7430 Rewrite the BlockScanner to use O(1) memory and use multiple threads
• is related to: HDFS-7548 Corrupt block reporting delayed until datablock scanner thread detects it
• relates to: HDFS-12514 Cancelled HedgedReads cause block to be marked as suspect on Windows