Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12402

Refactor ErasureCodingPolicyManager and related codes

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-alpha4
    • Fix Version/s: 3.0.0-beta1
    • Component/s: erasure-coding
    • Labels:
      None

      Description

      1. Correct message string grammar error
      2. Use HadoopIllegalArgumentException instead of IllegalECPolicyException
      3. Use HadoopIllegalArgumentException instead of IllegalArgumentException
      4. Remove IllegalECPolicyException

      1. HDFS-12402.001.patch
        5 kB
        SammiChen
      2. HDFS-12402.002.patch
        12 kB
        SammiChen

        Activity

        Hide
        Sammi SammiChen added a comment -

        Initial patch

        Show
        Sammi SammiChen added a comment - Initial patch
        Hide
        rakeshr Rakesh R added a comment -

        Thanks SammiChen for the patch. It looks like the patch has modified HadoopIllegalArgumentException and IllegalECPolicyException is still unchanged.

        Show
        rakeshr Rakesh R added a comment - Thanks SammiChen for the patch. It looks like the patch has modified HadoopIllegalArgumentException and IllegalECPolicyException is still unchanged.
        Hide
        Sammi SammiChen added a comment -

        Yes, Rakesh R, thanks for the reminder. I will upload a new patch.

        Show
        Sammi SammiChen added a comment - Yes, Rakesh R , thanks for the reminder. I will upload a new patch.
        Hide
        Sammi SammiChen added a comment -

        Patch refined.

        Show
        Sammi SammiChen added a comment - Patch refined.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 14s Docker mode activated.
              Prechecks
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
              trunk Compile Tests
        +1 mvninstall 13m 30s trunk passed
        +1 compile 0m 48s trunk passed
        +1 checkstyle 0m 36s trunk passed
        +1 mvnsite 0m 54s trunk passed
        +1 findbugs 1m 38s trunk passed
        +1 javadoc 0m 41s trunk passed
              Patch Compile Tests
        +1 mvninstall 0m 48s the patch passed
        +1 compile 0m 46s the patch passed
        +1 javac 0m 46s the patch passed
        +1 checkstyle 0m 33s the patch passed
        +1 mvnsite 0m 51s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 46s the patch passed
        +1 javadoc 0m 39s the patch passed
              Other Tests
        -1 unit 93m 18s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 16s The patch does not generate ASF License warnings.
        118m 37s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes
          hadoop.hdfs.TestLeaseRecoveryStriped
          hadoop.hdfs.TestAclsEndToEnd
          hadoop.hdfs.TestPipelines
          hadoop.hdfs.TestDFSStripedOutputStreamWithFailure070
        Timed out junit tests org.apache.hadoop.hdfs.TestWriteReadStripedFile



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:71bbb86
        JIRA Issue HDFS-12402
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12885735/HDFS-12402.001.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 89229242ea38 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / b6e7d13
        Default Java 1.8.0_144
        findbugs v3.1.0-RC1
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/21031/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/21031/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/21031/console
        Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.       trunk Compile Tests +1 mvninstall 13m 30s trunk passed +1 compile 0m 48s trunk passed +1 checkstyle 0m 36s trunk passed +1 mvnsite 0m 54s trunk passed +1 findbugs 1m 38s trunk passed +1 javadoc 0m 41s trunk passed       Patch Compile Tests +1 mvninstall 0m 48s the patch passed +1 compile 0m 46s the patch passed +1 javac 0m 46s the patch passed +1 checkstyle 0m 33s the patch passed +1 mvnsite 0m 51s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 46s the patch passed +1 javadoc 0m 39s the patch passed       Other Tests -1 unit 93m 18s hadoop-hdfs in the patch failed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 118m 37s Reason Tests Failed junit tests hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes   hadoop.hdfs.TestLeaseRecoveryStriped   hadoop.hdfs.TestAclsEndToEnd   hadoop.hdfs.TestPipelines   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure070 Timed out junit tests org.apache.hadoop.hdfs.TestWriteReadStripedFile Subsystem Report/Notes Docker Image:yetus/hadoop:71bbb86 JIRA Issue HDFS-12402 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12885735/HDFS-12402.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 89229242ea38 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / b6e7d13 Default Java 1.8.0_144 findbugs v3.1.0-RC1 unit https://builds.apache.org/job/PreCommit-HDFS-Build/21031/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/21031/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/21031/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 16s Docker mode activated.
              Prechecks
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
              trunk Compile Tests
        0 mvndep 0m 14s Maven dependency ordering for branch
        +1 mvninstall 13m 46s trunk passed
        +1 compile 1m 26s trunk passed
        +1 checkstyle 0m 43s trunk passed
        +1 mvnsite 1m 28s trunk passed
        +1 findbugs 3m 6s trunk passed
        +1 javadoc 1m 2s trunk passed
              Patch Compile Tests
        0 mvndep 0m 8s Maven dependency ordering for patch
        +1 mvninstall 1m 22s the patch passed
        +1 compile 1m 23s the patch passed
        +1 javac 1m 23s the patch passed
        +1 checkstyle 0m 41s the patch passed
        +1 mvnsite 1m 23s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 3m 13s the patch passed
        +1 javadoc 0m 56s the patch passed
              Other Tests
        +1 unit 1m 13s hadoop-hdfs-client in the patch passed.
        -1 unit 89m 47s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 16s The patch does not generate ASF License warnings.
        123m 47s



        Reason Tests
        Failed junit tests hadoop.hdfs.TestBlockStoragePolicy
          hadoop.hdfs.TestClientProtocolForPipelineRecovery
          hadoop.hdfs.TestLeaseRecoveryStriped
          hadoop.hdfs.TestEncryptionZonesWithHA
          hadoop.hdfs.TestDFSStripedOutputStreamWithFailure170
        Timed out junit tests org.apache.hadoop.hdfs.TestWriteReadStripedFile



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:71bbb86
        JIRA Issue HDFS-12402
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12885749/HDFS-12402.002.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 0ec323238a3e 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / b6e7d13
        Default Java 1.8.0_144
        findbugs v3.1.0-RC1
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/21034/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/21034/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/21034/console
        Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.       trunk Compile Tests 0 mvndep 0m 14s Maven dependency ordering for branch +1 mvninstall 13m 46s trunk passed +1 compile 1m 26s trunk passed +1 checkstyle 0m 43s trunk passed +1 mvnsite 1m 28s trunk passed +1 findbugs 3m 6s trunk passed +1 javadoc 1m 2s trunk passed       Patch Compile Tests 0 mvndep 0m 8s Maven dependency ordering for patch +1 mvninstall 1m 22s the patch passed +1 compile 1m 23s the patch passed +1 javac 1m 23s the patch passed +1 checkstyle 0m 41s the patch passed +1 mvnsite 1m 23s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 13s the patch passed +1 javadoc 0m 56s the patch passed       Other Tests +1 unit 1m 13s hadoop-hdfs-client in the patch passed. -1 unit 89m 47s hadoop-hdfs in the patch failed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 123m 47s Reason Tests Failed junit tests hadoop.hdfs.TestBlockStoragePolicy   hadoop.hdfs.TestClientProtocolForPipelineRecovery   hadoop.hdfs.TestLeaseRecoveryStriped   hadoop.hdfs.TestEncryptionZonesWithHA   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure170 Timed out junit tests org.apache.hadoop.hdfs.TestWriteReadStripedFile Subsystem Report/Notes Docker Image:yetus/hadoop:71bbb86 JIRA Issue HDFS-12402 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12885749/HDFS-12402.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 0ec323238a3e 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / b6e7d13 Default Java 1.8.0_144 findbugs v3.1.0-RC1 unit https://builds.apache.org/job/PreCommit-HDFS-Build/21034/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/21034/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/21034/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        drankye Kai Zheng added a comment -

        The patch LGTM and +1. Will commit it shortly.

        Show
        drankye Kai Zheng added a comment - The patch LGTM and +1. Will commit it shortly.
        Hide
        drankye Kai Zheng added a comment -

        Thanks Sammi for the contribution and Rakesh for the review.

        Committed to trunk and branch-3.0.

        Show
        drankye Kai Zheng added a comment - Thanks Sammi for the contribution and Rakesh for the review. Committed to trunk and branch-3.0.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12811 (See https://builds.apache.org/job/Hadoop-trunk-Commit/12811/)
        HDFS-12402. Refactor ErasureCodingPolicyManager and related codes. (kai.zheng: rev 2adf8bed712e6d770a0d53eea198d8911ae1a258)

        • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/AddECPolicyResponse.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ErasureCodingPolicyManager.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDistributedFileSystem.java
        • (delete) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/IllegalECPolicyException.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirErasureCodingOp.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12811 (See https://builds.apache.org/job/Hadoop-trunk-Commit/12811/ ) HDFS-12402 . Refactor ErasureCodingPolicyManager and related codes. (kai.zheng: rev 2adf8bed712e6d770a0d53eea198d8911ae1a258) (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/AddECPolicyResponse.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ErasureCodingPolicyManager.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDistributedFileSystem.java (delete) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/IllegalECPolicyException.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirErasureCodingOp.java

          People

          • Assignee:
            Sammi SammiChen
            Reporter:
            Sammi SammiChen
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development