Discovery of corrupt block reported in name node log

• Key: HADOOP-3193
• Type: Improvement
• Status: Closed
• Resolution: Fixed
• Priority: Minor
• Assignee: Chris Douglas
• Reporter: Robert Chansler
• Fix Version/s: 0.18.0

Description

Any discovery of a corrupt/unreadable block must be reported in the name node log.

Comments

[dhruba borthakur]

When a client discovers a corrupt block, it reports it to the namenode. The Namenode logs a "ReportBadBlock message" it in the namenode log. One improvement would be to enhance the log message to print the blockId(s) as well!

Another improvement would be to report the number of corrupted blocks through the HadoopMetrics API.

[dhruba borthakur]

I am marking this as an incompatible change, especially because a new HadoopMetric config file needs to be deployed to existing clusters to display "BlocksCorrupted".

[Tsz Wo (Nicholas), SZE]

What are the cases that a client (non-datanode client) should call reportBadBlocks(...)? I am concerned about the security issue.

[Lohit Vijayarenu]

+1 patch looks good. One small thing, the metric seem to report number of corrupt blocks reported over time. Should it be changed to number of corrupt blocks in the system at any point of time, possibly using MetricsIntValue. And also, namesystem.markBlockAsCorrupt logs a message about this inside corruptReplicas.addToCorruptReplicasMap function.

[Chris Douglas]

Lohit is right about the logging; it's redundant since HADOOP-2065.

Canceling this patch until we decide what to do with the metric.

[Sameer Paranjpye]

I'd like to do more here. In addition to reporting corrupt blocks in the log, the Namenode should try and determine where the corruption occured i.e. on disk on the Datanode vs elsewhere (network transmission or in memory on the client).

[Chris Douglas]

Revised to include Lohit's feedback

[Lohit Vijayarenu]

+1 Patch looks good

[Chris Douglas]

Fixed findbugs warning

[Chris Douglas]

     [exec] -1 overall.  

     [exec]     +1 @author.  The patch does not contain any @author tags.

     [exec]     -1 tests included.  The patch doesn't appear to include any new or modified tests.
     [exec]                         Please justify why no tests are needed for this patch.

     [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.

     [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

     [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.

No tests are included, as the change is only to logging and adding a metric.

[ edit - all dfs tests pass on my machine ]

[Chris Douglas]

In the future, it would be helpful if we included not only where the error occured, but more details about the particular error. Created HADOOP-3510 to track this improvement.

[Chris Douglas]

I just committed this.