Issue Details (XML | Word | Printable)

Key: HADOOP-3013
Type: Improvement Improvement
Status: Resolved Resolved
Resolution: Fixed
Priority: Major Major
Assignee: Lohit Vijayarenu
Reporter: Koji Noguchi
Votes: 0
Watchers: 0
Operations

If you were logged in you would be able to see more operations.
Hadoop Common

fsck to show (checksum) corrupted files

Created: 13/Mar/08 10:04 PM   Updated: 08/Jul/09 04:42 PM
Component/s: None
Affects Version/s: None
Fix Version/s: 0.18.0

Time Tracking:
Not Specified

File Attachments:
  File Name Date Attached Attached By Size
Text File Licensed for inclusion in ASF works HADOOP-3013-1.patch 2008-05-15 09:21 AM Lohit Vijayarenu 7 kB
Issue Links:
Reference
 

Hadoop Flags: Reviewed
Release Note: fsck reports corrupt blocks in the system.
Resolution Date: 15/May/08 08:55 PM


 Description  « Hide
Currently, only way to find files with all replica being corrupt is when we read those files.

Instead, can we have fsck report those?
(Using the corrupted blocks found by the periodic verification...?)

 All   Comments   Work Log   Change History   Subversion Commits      Sort Order: Ascending order - Click to sort in descending order
[ Permlink | « Hide ]
dhruba borthakur added a comment - 13/Mar/08 10:08 PM
We can enhance the Datanode Block verifier to persistently remember corrupted blocks. This information could be collected by the namenode (through block reports).

[ Permlink | « Hide ]
Lohit Vijayarenu added a comment - 15/May/08 09:21 AM
Attached patch now reports corrupt blocks via fsck. I think we should have not special option, but rather show them by default as done for MISSING blocks. Included test case to check the report of such blocks.

[ Permlink | « Hide ]
Hadoop QA added a comment - 15/May/08 10:48 AM
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12382096/HADOOP-3013-1.patch
against trunk revision 656525.

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 4 new or modified tests.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac compiler warnings.

+1 findbugs. The patch does not introduce any new Findbugs warnings.

+1 release audit. The applied patch does not increase the total number of release audit warnings.

+1 core tests. The patch passed core unit tests.

+1 contrib tests. The patch passed contrib unit tests.

Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2478/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2478/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2478/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2478/console

This message is automatically generated.


[ Permlink | « Hide ]
Raghu Angadi added a comment - 15/May/08 06:20 PM
+1. Patch looks good.

[ Permlink | « Hide ]
Chris Douglas added a comment - 15/May/08 08:55 PM
I just committed this. Thanks, Lohit

[ Permlink | « Hide ]
Hudson added a comment - 16/May/08 12:52 PM
Integrated in Hadoop-trunk #493 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/493/)


Powered by a free Atlassian JIRA open source license for Apache Software Foundation. Try JIRA - bug tracking software for your team.
Atlassian JIRA the Professional Issue Tracker. (Enterprise Edition, Version: 3.13.2-#335) - Bug/feature request - Atlassian news - Contact Administrators