[HDFS-1111] getCorruptFiles() should give some hint that the list is not complete

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.22.0
• Fix Version/s: 0.22.0
• Hadoop Flags: Reviewed
• Assignee: Sriram Rao
• Reporter: Rodrigo Schmidt
• Created: 26/Apr/10 20:38
• Updated: 02/May/13 02:29
• Resolved: 11/Sep/10 21:55

Description

If the list of corruptfiles returned by the namenode doesn't say anything if the number of corrupted files is larger than the call output limit (which means the list is not complete). There should be a way to hint incompleteness to clients.

A simple hack would be to add an extra entry to the array returned with the value null. Clients could interpret this as a sign that there are other corrupt files in the system.

We should also do some rephrasing of the fsck output to make it more confident when the list is not complete and less confident when the list is known to be incomplete.

Issue Links

• depends upon: HDFS-1032 Extend DFSck with an option to list corrupt files using API from HDFS-729
• depends upon: HDFS-729 fsck option to list only corrupted files
• is blocked by: HADOOP-6743 Create a Writable class that stores a possibly incomplete list of FileStatus objects
• is related to: HDFS-1265 Change the getCorruptFiles API so that caller can specify directory it wants to look into.
• is related to: HDFS-1031 Enhance the webUi to list a few of the corrupted files in HDFS