Hadoop Common
  1. Hadoop Common
  2. HADOOP-7060

A more elegant FileSystem#listCorruptFileBlocks API

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.0
    • Component/s: fs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      I'd like to change the newly added listCorruptFileBlocks signature to be:

      /**
      * Get all files with corrupt blocks under the given path
      */
      RemoteIterator<Path> listCorruptFileBlocks(Path src) throws IOException;
      

      This new API does not expose "cookie" to user although underlying implementation may still need to invoke multiple RPCs to get the whole list.

      1. HADOOP-7060.patch
        5 kB
        Patrick Kling
      2. HADOOP-7060.2.patch
        10 kB
        Patrick Kling

        Issue Links

          Activity

          Hide
          Patrick Kling added a comment -

          ant test passes.

          ant test-patch results:

               [exec] -1 overall.  
               [exec] 
               [exec]     +1 @author.  The patch does not contain any @author tags.
               [exec] 
               [exec]     -1 tests included.  The patch doesn't appear to include any new or modified tests.
               [exec]                         Please justify why no new tests are needed for this patch.
               [exec]                         Also please list what manual steps were performed to verify this patch.
               [exec] 
               [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
               [exec] 
               [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
               [exec] 
               [exec]     +1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) warnings.
               [exec] 
               [exec]     +1 release audit.  The applied patch does not increase the total number of release audit warnings.
               [exec] 
               [exec]     +1 system test framework.  The patch passed system test framework compile.
          

          A test for this change is included in HDFS-1533 (TestListCorruptFileBlocks).

          Show
          Patrick Kling added a comment - ant test passes. ant test-patch results: [exec] -1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] -1 tests included. The patch doesn't appear to include any new or modified tests. [exec] Please justify why no new tests are needed for this patch. [exec] Also please list what manual steps were performed to verify this patch. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] +1 system test framework. The patch passed system test framework compile. A test for this change is included in HDFS-1533 (TestListCorruptFileBlocks).
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12465880/HADOOP-7060.patch
          against trunk revision 1043117.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/136//testReport/
          Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/136//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/136//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12465880/HADOOP-7060.patch against trunk revision 1043117. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/136//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/136//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/136//console This message is automatically generated.
          Hide
          Hairong Kuang added a comment -

          Please remove class CorruptFileBlocks. We need to move it to HDFS.

          Show
          Hairong Kuang added a comment - Please remove class CorruptFileBlocks. We need to move it to HDFS.
          Hide
          Patrick Kling added a comment -

          Moved CorruptFileBlocks (and TestCorruptFileBlocks) to HDFS.

          Test/test-patch results are unchanged.

          Show
          Patrick Kling added a comment - Moved CorruptFileBlocks (and TestCorruptFileBlocks) to HDFS. Test/test-patch results are unchanged.
          Hide
          Patrick Kling added a comment -

          Actually, ant test-patch is +1 overall now since deleting a test counts as modifying it...

          Show
          Patrick Kling added a comment - Actually, ant test-patch is +1 overall now since deleting a test counts as modifying it...
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12465934/HADOOP-7060.2.patch
          against trunk revision 1043117.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/138//testReport/
          Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/138//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/138//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12465934/HADOOP-7060.2.patch against trunk revision 1043117. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/138//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/138//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/138//console This message is automatically generated.
          Hide
          Hairong Kuang added a comment -

          +1. This looks good to me.

          Show
          Hairong Kuang added a comment - +1. This looks good to me.
          Hide
          Hairong Kuang added a comment -

          I've just committed this.

          Show
          Hairong Kuang added a comment - I've just committed this.
          Hide
          Hairong Kuang added a comment -

          Thanks, Patrick!

          Show
          Hairong Kuang added a comment - Thanks, Patrick!
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #454 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/454/)
          HADOOP-7060 Remove the unused files from the repository

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #454 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/454/ ) HADOOP-7060 Remove the unused files from the repository
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk #541 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/541/)
          HADOOP-7060 Remove the unused files from the repository
          HADOOP-7060. A more elegant FileSystem#listCorruptFileBlocks API. Contributed by Patrick Kling.

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk #541 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/541/ ) HADOOP-7060 Remove the unused files from the repository HADOOP-7060 . A more elegant FileSystem#listCorruptFileBlocks API. Contributed by Patrick Kling.

            People

            • Assignee:
              Patrick Kling
              Reporter:
              Hairong Kuang
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development