Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12402

Refactor ErasureCodingPolicyManager and related codes

        Details

        • Type: Improvement
        • Status: Resolved
        • Priority: Major
        • Resolution: Fixed
        • Affects Version/s: 3.0.0-alpha4
        • Fix Version/s: 3.0.0-beta1
        • Component/s: erasure-coding
        • Labels:
          None

          Description

          1. Correct message string grammar error
          2. Use HadoopIllegalArgumentException instead of IllegalECPolicyException
          3. Use HadoopIllegalArgumentException instead of IllegalArgumentException
          4. Remove IllegalECPolicyException

          1. Text File
            HDFS-12402.001.patch
            5 kB
            SammiChen
          2. Text File
            HDFS-12402.002.patch
            12 kB
            SammiChen

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            Sammi SammiChen added a comment -

            Initial patch

            Show
            Sammi SammiChen added a comment - Initial patch
            Hide
            Permalink
            rakeshr Rakesh R added a comment -

            Thanks SammiChen for the patch. It looks like the patch has modified HadoopIllegalArgumentException and IllegalECPolicyException is still unchanged.

            Show
            rakeshr Rakesh R added a comment - Thanks SammiChen for the patch. It looks like the patch has modified HadoopIllegalArgumentException and IllegalECPolicyException is still unchanged.
            Hide
            Permalink
            Sammi SammiChen added a comment -

            Yes, Rakesh R, thanks for the reminder. I will upload a new patch.

            Show
            Sammi SammiChen added a comment - Yes, Rakesh R , thanks for the reminder. I will upload a new patch.
            Hide
            Permalink
            Sammi SammiChen added a comment -

            Patch refined.

            Show
            Sammi SammiChen added a comment - Patch refined.
            Hide
            Permalink
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 14s Docker mode activated.
                  Prechecks
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
                  trunk Compile Tests
            +1 mvninstall 13m 30s trunk passed
            +1 compile 0m 48s trunk passed
            +1 checkstyle 0m 36s trunk passed
            +1 mvnsite 0m 54s trunk passed
            +1 findbugs 1m 38s trunk passed
            +1 javadoc 0m 41s trunk passed
                  Patch Compile Tests
            +1 mvninstall 0m 48s the patch passed
            +1 compile 0m 46s the patch passed
            +1 javac 0m 46s the patch passed
            +1 checkstyle 0m 33s the patch passed
            +1 mvnsite 0m 51s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 findbugs 1m 46s the patch passed
            +1 javadoc 0m 39s the patch passed
                  Other Tests
            -1 unit 93m 18s hadoop-hdfs in the patch failed.
            +1 asflicense 0m 16s The patch does not generate ASF License warnings.
            118m 37s



            Reason Tests
            Failed junit tests hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes
              hadoop.hdfs.TestLeaseRecoveryStriped
              hadoop.hdfs.TestAclsEndToEnd
              hadoop.hdfs.TestPipelines
              hadoop.hdfs.TestDFSStripedOutputStreamWithFailure070
            Timed out junit tests org.apache.hadoop.hdfs.TestWriteReadStripedFile



            Subsystem Report/Notes
            Docker Image:yetus/hadoop:71bbb86
            JIRA Issue HDFS-12402
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12885735/HDFS-12402.001.patch
            Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
            uname Linux 89229242ea38 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
            git revision trunk / b6e7d13
            Default Java 1.8.0_144
            findbugs v3.1.0-RC1
            unit https://builds.apache.org/job/PreCommit-HDFS-Build/21031/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
            Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/21031/testReport/
            modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
            Console output https://builds.apache.org/job/PreCommit-HDFS-Build/21031/console
            Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

            This message was automatically generated.

            Show
            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.       trunk Compile Tests +1 mvninstall 13m 30s trunk passed +1 compile 0m 48s trunk passed +1 checkstyle 0m 36s trunk passed +1 mvnsite 0m 54s trunk passed +1 findbugs 1m 38s trunk passed +1 javadoc 0m 41s trunk passed       Patch Compile Tests +1 mvninstall 0m 48s the patch passed +1 compile 0m 46s the patch passed +1 javac 0m 46s the patch passed +1 checkstyle 0m 33s the patch passed +1 mvnsite 0m 51s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 46s the patch passed +1 javadoc 0m 39s the patch passed       Other Tests -1 unit 93m 18s hadoop-hdfs in the patch failed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 118m 37s Reason Tests Failed junit tests hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes   hadoop.hdfs.TestLeaseRecoveryStriped   hadoop.hdfs.TestAclsEndToEnd   hadoop.hdfs.TestPipelines   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure070 Timed out junit tests org.apache.hadoop.hdfs.TestWriteReadStripedFile Subsystem Report/Notes Docker Image:yetus/hadoop:71bbb86 JIRA Issue HDFS-12402 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12885735/HDFS-12402.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 89229242ea38 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / b6e7d13 Default Java 1.8.0_144 findbugs v3.1.0-RC1 unit https://builds.apache.org/job/PreCommit-HDFS-Build/21031/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/21031/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/21031/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
            Hide
            Permalink
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 16s Docker mode activated.
                  Prechecks
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
                  trunk Compile Tests
            0 mvndep 0m 14s Maven dependency ordering for branch
            +1 mvninstall 13m 46s trunk passed
            +1 compile 1m 26s trunk passed
            +1 checkstyle 0m 43s trunk passed
            +1 mvnsite 1m 28s trunk passed
            +1 findbugs 3m 6s trunk passed
            +1 javadoc 1m 2s trunk passed
                  Patch Compile Tests
            0 mvndep 0m 8s Maven dependency ordering for patch
            +1 mvninstall 1m 22s the patch passed
            +1 compile 1m 23s the patch passed
            +1 javac 1m 23s the patch passed
            +1 checkstyle 0m 41s the patch passed
            +1 mvnsite 1m 23s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 findbugs 3m 13s the patch passed
            +1 javadoc 0m 56s the patch passed
                  Other Tests
            +1 unit 1m 13s hadoop-hdfs-client in the patch passed.
            -1 unit 89m 47s hadoop-hdfs in the patch failed.
            +1 asflicense 0m 16s The patch does not generate ASF License warnings.
            123m 47s



            Reason Tests
            Failed junit tests hadoop.hdfs.TestBlockStoragePolicy
              hadoop.hdfs.TestClientProtocolForPipelineRecovery
              hadoop.hdfs.TestLeaseRecoveryStriped
              hadoop.hdfs.TestEncryptionZonesWithHA
              hadoop.hdfs.TestDFSStripedOutputStreamWithFailure170
            Timed out junit tests org.apache.hadoop.hdfs.TestWriteReadStripedFile



            Subsystem Report/Notes
            Docker Image:yetus/hadoop:71bbb86
            JIRA Issue HDFS-12402
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12885749/HDFS-12402.002.patch
            Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
            uname Linux 0ec323238a3e 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
            git revision trunk / b6e7d13
            Default Java 1.8.0_144
            findbugs v3.1.0-RC1
            unit https://builds.apache.org/job/PreCommit-HDFS-Build/21034/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
            Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/21034/testReport/
            modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
            Console output https://builds.apache.org/job/PreCommit-HDFS-Build/21034/console
            Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

            This message was automatically generated.

            Show
            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.       trunk Compile Tests 0 mvndep 0m 14s Maven dependency ordering for branch +1 mvninstall 13m 46s trunk passed +1 compile 1m 26s trunk passed +1 checkstyle 0m 43s trunk passed +1 mvnsite 1m 28s trunk passed +1 findbugs 3m 6s trunk passed +1 javadoc 1m 2s trunk passed       Patch Compile Tests 0 mvndep 0m 8s Maven dependency ordering for patch +1 mvninstall 1m 22s the patch passed +1 compile 1m 23s the patch passed +1 javac 1m 23s the patch passed +1 checkstyle 0m 41s the patch passed +1 mvnsite 1m 23s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 13s the patch passed +1 javadoc 0m 56s the patch passed       Other Tests +1 unit 1m 13s hadoop-hdfs-client in the patch passed. -1 unit 89m 47s hadoop-hdfs in the patch failed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 123m 47s Reason Tests Failed junit tests hadoop.hdfs.TestBlockStoragePolicy   hadoop.hdfs.TestClientProtocolForPipelineRecovery   hadoop.hdfs.TestLeaseRecoveryStriped   hadoop.hdfs.TestEncryptionZonesWithHA   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure170 Timed out junit tests org.apache.hadoop.hdfs.TestWriteReadStripedFile Subsystem Report/Notes Docker Image:yetus/hadoop:71bbb86 JIRA Issue HDFS-12402 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12885749/HDFS-12402.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 0ec323238a3e 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / b6e7d13 Default Java 1.8.0_144 findbugs v3.1.0-RC1 unit https://builds.apache.org/job/PreCommit-HDFS-Build/21034/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/21034/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/21034/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
            Hide
            Permalink
            drankye Kai Zheng added a comment -

            The patch LGTM and +1. Will commit it shortly.

            Show
            drankye Kai Zheng added a comment - The patch LGTM and +1. Will commit it shortly.
            Hide
            Permalink
            drankye Kai Zheng added a comment -

            Thanks Sammi for the contribution and Rakesh for the review.

            Committed to trunk and branch-3.0.

            Show
            drankye Kai Zheng added a comment - Thanks Sammi for the contribution and Rakesh for the review. Committed to trunk and branch-3.0.
            Hide
            Permalink
            hudson Hudson added a comment -

            SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12811 (See https://builds.apache.org/job/Hadoop-trunk-Commit/12811/)
            HDFS-12402. Refactor ErasureCodingPolicyManager and related codes. (kai.zheng: rev 2adf8bed712e6d770a0d53eea198d8911ae1a258)

            • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/AddECPolicyResponse.java
            • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
            • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ErasureCodingPolicyManager.java
            • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDistributedFileSystem.java
            • (delete) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/IllegalECPolicyException.java
            • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirErasureCodingOp.java
            Show
            hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12811 (See https://builds.apache.org/job/Hadoop-trunk-Commit/12811/ ) HDFS-12402 . Refactor ErasureCodingPolicyManager and related codes. (kai.zheng: rev 2adf8bed712e6d770a0d53eea198d8911ae1a258) (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/AddECPolicyResponse.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ErasureCodingPolicyManager.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDistributedFileSystem.java (delete) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/IllegalECPolicyException.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirErasureCodingOp.java

              People

              • Assignee:
                Sammi SammiChen
                Reporter:
                Sammi SammiChen
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Development