Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-7060

A more elegant FileSystem#listCorruptFileBlocks API

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.23.0
    • 0.23.0
    • fs
    • None
    • Reviewed

    Description

      I'd like to change the newly added listCorruptFileBlocks signature to be:

      /**
      * Get all files with corrupt blocks under the given path
      */
      RemoteIterator<Path> listCorruptFileBlocks(Path src) throws IOException;
      

      This new API does not expose "cookie" to user although underlying implementation may still need to invoke multiple RPCs to get the whole list.

      Attachments

        1. HADOOP-7060.2.patch
          10 kB
          Patrick Kling
        2. HADOOP-7060.patch
          5 kB
          Patrick Kling

        Issue Links

          Activity

            pkling Patrick Kling added a comment -

            ant test passes.

            ant test-patch results:

                 [exec] -1 overall.  
                 [exec] 
                 [exec]     +1 @author.  The patch does not contain any @author tags.
                 [exec] 
                 [exec]     -1 tests included.  The patch doesn't appear to include any new or modified tests.
                 [exec]                         Please justify why no new tests are needed for this patch.
                 [exec]                         Also please list what manual steps were performed to verify this patch.
                 [exec] 
                 [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
                 [exec] 
                 [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
                 [exec] 
                 [exec]     +1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) warnings.
                 [exec] 
                 [exec]     +1 release audit.  The applied patch does not increase the total number of release audit warnings.
                 [exec] 
                 [exec]     +1 system test framework.  The patch passed system test framework compile.
            

            A test for this change is included in HDFS-1533 (TestListCorruptFileBlocks).

            pkling Patrick Kling added a comment - ant test passes. ant test-patch results: [exec] -1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] -1 tests included. The patch doesn't appear to include any new or modified tests. [exec] Please justify why no new tests are needed for this patch. [exec] Also please list what manual steps were performed to verify this patch. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] +1 system test framework. The patch passed system test framework compile. A test for this change is included in HDFS-1533 (TestListCorruptFileBlocks).
            hadoopqa Hadoop QA added a comment -

            -1 overall. Here are the results of testing the latest attachment
            http://issues.apache.org/jira/secure/attachment/12465880/HADOOP-7060.patch
            against trunk revision 1043117.

            +1 @author. The patch does not contain any @author tags.

            -1 tests included. The patch doesn't appear to include any new or modified tests.
            Please justify why no new tests are needed for this patch.
            Also please list what manual steps were performed to verify this patch.

            +1 javadoc. The javadoc tool did not generate any warning messages.

            +1 javac. The applied patch does not increase the total number of javac compiler warnings.

            +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

            +1 release audit. The applied patch does not increase the total number of release audit warnings.

            +1 core tests. The patch passed core unit tests.

            +1 contrib tests. The patch passed contrib unit tests.

            +1 system test framework. The patch passed system test framework compile.

            Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/136//testReport/
            Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/136//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
            Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/136//console

            This message is automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12465880/HADOOP-7060.patch against trunk revision 1043117. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/136//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/136//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/136//console This message is automatically generated.
            hairong Hairong Kuang added a comment -

            Please remove class CorruptFileBlocks. We need to move it to HDFS.

            hairong Hairong Kuang added a comment - Please remove class CorruptFileBlocks. We need to move it to HDFS.
            pkling Patrick Kling added a comment -

            Moved CorruptFileBlocks (and TestCorruptFileBlocks) to HDFS.

            Test/test-patch results are unchanged.

            pkling Patrick Kling added a comment - Moved CorruptFileBlocks (and TestCorruptFileBlocks) to HDFS. Test/test-patch results are unchanged.
            pkling Patrick Kling added a comment -

            Actually, ant test-patch is +1 overall now since deleting a test counts as modifying it...

            pkling Patrick Kling added a comment - Actually, ant test-patch is +1 overall now since deleting a test counts as modifying it...
            hadoopqa Hadoop QA added a comment -

            +1 overall. Here are the results of testing the latest attachment
            http://issues.apache.org/jira/secure/attachment/12465934/HADOOP-7060.2.patch
            against trunk revision 1043117.

            +1 @author. The patch does not contain any @author tags.

            +1 tests included. The patch appears to include 3 new or modified tests.

            +1 javadoc. The javadoc tool did not generate any warning messages.

            +1 javac. The applied patch does not increase the total number of javac compiler warnings.

            +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

            +1 release audit. The applied patch does not increase the total number of release audit warnings.

            +1 core tests. The patch passed core unit tests.

            +1 contrib tests. The patch passed contrib unit tests.

            +1 system test framework. The patch passed system test framework compile.

            Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/138//testReport/
            Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/138//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
            Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/138//console

            This message is automatically generated.

            hadoopqa Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12465934/HADOOP-7060.2.patch against trunk revision 1043117. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/138//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/138//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/138//console This message is automatically generated.
            hairong Hairong Kuang added a comment -

            +1. This looks good to me.

            hairong Hairong Kuang added a comment - +1. This looks good to me.
            hairong Hairong Kuang added a comment -

            I've just committed this.

            hairong Hairong Kuang added a comment - I've just committed this.
            hairong Hairong Kuang added a comment -

            Thanks, Patrick!

            hairong Hairong Kuang added a comment - Thanks, Patrick!
            hudson Hudson added a comment -

            Integrated in Hadoop-Common-trunk-Commit #454 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/454/)
            HADOOP-7060 Remove the unused files from the repository

            hudson Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #454 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/454/ ) HADOOP-7060 Remove the unused files from the repository
            hudson Hudson added a comment -

            Integrated in Hadoop-Common-trunk #541 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/541/)
            HADOOP-7060 Remove the unused files from the repository
            HADOOP-7060. A more elegant FileSystem#listCorruptFileBlocks API. Contributed by Patrick Kling.

            hudson Hudson added a comment - Integrated in Hadoop-Common-trunk #541 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/541/ ) HADOOP-7060 Remove the unused files from the repository HADOOP-7060 . A more elegant FileSystem#listCorruptFileBlocks API. Contributed by Patrick Kling.

            People

              pkling Patrick Kling
              hairong Hairong Kuang
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: