[HDFS-14381] Add option to hdfs dfs to ignore corrupt blocks

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 3.2.0
• Component/s: tools
• Assignee: Unassigned
• Reporter: Daniel Templeton
• Created: 19/Mar/19 15:37
• Updated: 19/Mar/19 17:49

Description

If I have a file in HDFS that contains 100 blocks, and I happen to lose the first block (for whatever obscure/unlikely/dumb reason), I can no longer access the 99% of the file that's still there and accessible. In the case of some data formats (e.g. text), the remaining data may still be useful. It would be nice to have a way to extract the remaining data without having to manually reassemble the file contents from the block files. Something like hdfs dfs -copyToLocal -ignoreCorrupt <file>. It could insert some marker to show where the missing blocks are.