Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      This method is very long, approximately 230 lines. Splitting this method into smaller methods.

      1. HDFS-8215.1.patch
        11 kB
        Takanobu Asanuma

        Issue Links

          Activity

          Hide
          tasanuma0829 Takanobu Asanuma added a comment -

          I created a first patch. There may be some task to do refactoring.

          1. The main change in this patch is creating some methods to reduce the length of Namenodefsck#check.
          2. I confirmed that TestFsck was passed.
          3. As an aside, I'm trying to support the fsck for Erasure Coding in HDFS-7687. I think we will create Namenodefsck#collectECBlockGroupsSummary, like Namenodefsck#collectBlocksSummary I created in this patch.

          Please review it. Thank you.

          Show
          tasanuma0829 Takanobu Asanuma added a comment - I created a first patch. There may be some task to do refactoring. The main change in this patch is creating some methods to reduce the length of Namenodefsck#check. I confirmed that TestFsck was passed. As an aside, I'm trying to support the fsck for Erasure Coding in HDFS-7687 . I think we will create Namenodefsck#collectECBlockGroupsSummary, like Namenodefsck#collectBlocksSummary I created in this patch. Please review it. Thank you.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 14m 33s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 javac 7m 27s There were no new javac warning messages.
          +1 javadoc 9m 38s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 6m 55s The applied patch generated 1 additional checkstyle issues.
          +1 install 1m 33s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 3m 6s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
          +1 native 3m 14s Pre-build of native portion
          -1 hdfs tests 175m 16s Tests failed in hadoop-hdfs.
              222m 44s  



          Reason Tests
          Failed unit tests hadoop.hdfs.TestDFSClientRetries



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12727591/HDFS-8215.1.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 189a63a
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/10360/artifact/patchprocess/checkstyle-result-diff.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/10360/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/10360/testReport/
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10360/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 33s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 javac 7m 27s There were no new javac warning messages. +1 javadoc 9m 38s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 6m 55s The applied patch generated 1 additional checkstyle issues. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 3m 6s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 native 3m 14s Pre-build of native portion -1 hdfs tests 175m 16s Tests failed in hadoop-hdfs.     222m 44s   Reason Tests Failed unit tests hadoop.hdfs.TestDFSClientRetries Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12727591/HDFS-8215.1.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 189a63a checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/10360/artifact/patchprocess/checkstyle-result-diff.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/10360/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/10360/testReport/ Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10360/console This message was automatically generated.
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          +1 the patch looks good. Thanks also for putting additional comments.

          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - +1 the patch looks good. Thanks also for putting additional comments.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #7654 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7654/)
          HDFS-8215. Refactor NamenodeFsck#check method. Contributed by Takanobu Asanuma (szetszwo: rev ef4e9963b25d7d2e30f1071ddcaa9d92a7fe70f3)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7654 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7654/ ) HDFS-8215 . Refactor NamenodeFsck#check method. Contributed by Takanobu Asanuma (szetszwo: rev ef4e9963b25d7d2e30f1071ddcaa9d92a7fe70f3) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          I have committed this. Thanks, Asanuma-san!

          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - I have committed this. Thanks, Asanuma-san!
          Hide
          tasanuma0829 Takanobu Asanuma added a comment -

          Thank you very much for the review and the commit, Nicholas!

          Show
          tasanuma0829 Takanobu Asanuma added a comment - Thank you very much for the review and the commit, Nicholas!
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2105 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2105/)
          HDFS-8215. Refactor NamenodeFsck#check method. Contributed by Takanobu Asanuma (szetszwo: rev ef4e9963b25d7d2e30f1071ddcaa9d92a7fe70f3)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2105 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2105/ ) HDFS-8215 . Refactor NamenodeFsck#check method. Contributed by Takanobu Asanuma (szetszwo: rev ef4e9963b25d7d2e30f1071ddcaa9d92a7fe70f3) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #164 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/164/)
          HDFS-8215. Refactor NamenodeFsck#check method. Contributed by Takanobu Asanuma (szetszwo: rev ef4e9963b25d7d2e30f1071ddcaa9d92a7fe70f3)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #164 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/164/ ) HDFS-8215 . Refactor NamenodeFsck#check method. Contributed by Takanobu Asanuma (szetszwo: rev ef4e9963b25d7d2e30f1071ddcaa9d92a7fe70f3) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #173 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/173/)
          HDFS-8215. Refactor NamenodeFsck#check method. Contributed by Takanobu Asanuma (szetszwo: rev ef4e9963b25d7d2e30f1071ddcaa9d92a7fe70f3)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #173 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/173/ ) HDFS-8215 . Refactor NamenodeFsck#check method. Contributed by Takanobu Asanuma (szetszwo: rev ef4e9963b25d7d2e30f1071ddcaa9d92a7fe70f3) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk #907 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/907/)
          HDFS-8215. Refactor NamenodeFsck#check method. Contributed by Takanobu Asanuma (szetszwo: rev ef4e9963b25d7d2e30f1071ddcaa9d92a7fe70f3)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #907 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/907/ ) HDFS-8215 . Refactor NamenodeFsck#check method. Contributed by Takanobu Asanuma (szetszwo: rev ef4e9963b25d7d2e30f1071ddcaa9d92a7fe70f3) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #174 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/174/)
          HDFS-8215. Refactor NamenodeFsck#check method. Contributed by Takanobu Asanuma (szetszwo: rev ef4e9963b25d7d2e30f1071ddcaa9d92a7fe70f3)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #174 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/174/ ) HDFS-8215 . Refactor NamenodeFsck#check method. Contributed by Takanobu Asanuma (szetszwo: rev ef4e9963b25d7d2e30f1071ddcaa9d92a7fe70f3) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2123 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2123/)
          HDFS-8215. Refactor NamenodeFsck#check method. Contributed by Takanobu Asanuma (szetszwo: rev ef4e9963b25d7d2e30f1071ddcaa9d92a7fe70f3)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2123 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2123/ ) HDFS-8215 . Refactor NamenodeFsck#check method. Contributed by Takanobu Asanuma (szetszwo: rev ef4e9963b25d7d2e30f1071ddcaa9d92a7fe70f3) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java

            People

            • Assignee:
              tasanuma0829 Takanobu Asanuma
              Reporter:
              tasanuma0829 Takanobu Asanuma
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development