Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11793

Allow to enable user defined erasure coding policy

    Details

      Description

      Dfs configuration key dfs.namenode.ec.policies.enabled support user defined erasure coding policy

      1. HDFS-11793.001.patch
        39 kB
        SammiChen
      2. HDFS-11793.002.patch
        43 kB
        SammiChen
      3. HDFS-11793.003.patch
        50 kB
        SammiChen
      4. HDFS-11793.004.patch
        52 kB
        SammiChen

        Activity

        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11771 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11771/)
        HDFS-11793. Allow to enable user defined erasure coding policy. (kai.zheng: rev a62be38a5e5d3a61dfb59054b3f5fd5b1b7053b3)

        • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/erasurecoding.proto
        • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/util/ECPolicyLoader.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/hdfs.proto
        • (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/erasurecode/TestCodecRegistry.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientNamenodeProtocolTranslatorPB.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
        • (delete) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/AddingECPolicyResponse.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/ErasureCodingPolicy.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirErasureCodingOp.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDistributedFileSystem.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSErasureCoding.md
        • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/ClientProtocol.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/conf/user_ec_policies.xml.template
        • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/CodecUtil.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/test/java/org/apache/hadoop/hdfs/protocol/TestErasureCodingPolicy.java
        • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/CodecRegistry.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
        • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelperClient.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/protocolPB/TestPBHelper.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ErasureCodingPolicyManager.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestErasureCodingPolicies.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/ECAdmin.java
        • (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/AddECPolicyResponse.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientNamenodeProtocolServerSideTranslatorPB.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11771 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11771/ ) HDFS-11793 . Allow to enable user defined erasure coding policy. (kai.zheng: rev a62be38a5e5d3a61dfb59054b3f5fd5b1b7053b3) (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/erasurecoding.proto (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/util/ECPolicyLoader.java (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/hdfs.proto (edit) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/erasurecode/TestCodecRegistry.java (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientNamenodeProtocolTranslatorPB.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java (delete) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/AddingECPolicyResponse.java (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/ErasureCodingPolicy.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirErasureCodingOp.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDistributedFileSystem.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSErasureCoding.md (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/ClientProtocol.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/conf/user_ec_policies.xml.template (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/CodecUtil.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/test/java/org/apache/hadoop/hdfs/protocol/TestErasureCodingPolicy.java (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/CodecRegistry.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelperClient.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/protocolPB/TestPBHelper.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ErasureCodingPolicyManager.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestErasureCodingPolicies.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/ECAdmin.java (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/AddECPolicyResponse.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientNamenodeProtocolServerSideTranslatorPB.java
        Hide
        drankye Kai Zheng added a comment -

        Committed to trunk. Thanks SammiChen for the contribution!

        Show
        drankye Kai Zheng added a comment - Committed to trunk. Thanks SammiChen for the contribution!
        Hide
        drankye Kai Zheng added a comment -

        The latest patch LGTM and +1. Will commit it shortly.

        Show
        drankye Kai Zheng added a comment - The latest patch LGTM and +1. Will commit it shortly.
        Hide
        Sammi SammiChen added a comment -

        Hi Andrew Wang, do you have time to take a look at this patch?

        Show
        Sammi SammiChen added a comment - Hi Andrew Wang , do you have time to take a look at this patch?
        Hide
        Sammi SammiChen added a comment -

        Double checked the failed unit test. They are irrelevant.

        Show
        Sammi SammiChen added a comment - Double checked the failed unit test. They are irrelevant.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 20s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
        0 mvndep 0m 16s Maven dependency ordering for branch
        +1 mvninstall 15m 25s trunk passed
        +1 compile 16m 13s trunk passed
        +1 checkstyle 2m 5s trunk passed
        +1 mvnsite 2m 50s trunk passed
        +1 mvneclipse 1m 3s trunk passed
        -1 findbugs 1m 39s hadoop-common-project/hadoop-common in trunk has 19 extant Findbugs warnings.
        -1 findbugs 1m 37s hadoop-hdfs-project/hadoop-hdfs-client in trunk has 2 extant Findbugs warnings.
        -1 findbugs 1m 58s hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant Findbugs warnings.
        +1 javadoc 2m 7s trunk passed
        0 mvndep 0m 15s Maven dependency ordering for patch
        +1 mvninstall 2m 20s the patch passed
        +1 compile 15m 46s the patch passed
        +1 cc 15m 46s the patch passed
        +1 javac 15m 46s the patch passed
        -0 checkstyle 2m 5s root: The patch generated 4 new + 973 unchanged - 0 fixed = 977 total (was 973)
        +1 mvnsite 2m 47s the patch passed
        +1 mvneclipse 1m 2s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 1s The patch has no ill-formed XML file.
        +1 findbugs 5m 6s the patch passed
        +1 javadoc 2m 5s the patch passed
        +1 unit 7m 55s hadoop-common in the patch passed.
        +1 unit 1m 23s hadoop-hdfs-client in the patch passed.
        -1 unit 67m 53s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 36s The patch does not generate ASF License warnings.
        156m 15s



        Reason Tests
        Failed junit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure070
          hadoop.hdfs.TestDFSRSDefault10x4StripedOutputStreamWithFailure



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:14b5c93
        JIRA Issue HDFS-11793
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12867513/HDFS-11793.004.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc xml
        uname Linux 793c6f0db906 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 51b671e
        Default Java 1.8.0_121
        findbugs v3.1.0-RC1
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19397/artifact/patchprocess/branch-findbugs-hadoop-common-project_hadoop-common-warnings.html
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19397/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19397/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19397/artifact/patchprocess/diff-checkstyle-root.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/19397/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19397/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19397/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. 0 mvndep 0m 16s Maven dependency ordering for branch +1 mvninstall 15m 25s trunk passed +1 compile 16m 13s trunk passed +1 checkstyle 2m 5s trunk passed +1 mvnsite 2m 50s trunk passed +1 mvneclipse 1m 3s trunk passed -1 findbugs 1m 39s hadoop-common-project/hadoop-common in trunk has 19 extant Findbugs warnings. -1 findbugs 1m 37s hadoop-hdfs-project/hadoop-hdfs-client in trunk has 2 extant Findbugs warnings. -1 findbugs 1m 58s hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant Findbugs warnings. +1 javadoc 2m 7s trunk passed 0 mvndep 0m 15s Maven dependency ordering for patch +1 mvninstall 2m 20s the patch passed +1 compile 15m 46s the patch passed +1 cc 15m 46s the patch passed +1 javac 15m 46s the patch passed -0 checkstyle 2m 5s root: The patch generated 4 new + 973 unchanged - 0 fixed = 977 total (was 973) +1 mvnsite 2m 47s the patch passed +1 mvneclipse 1m 2s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 5m 6s the patch passed +1 javadoc 2m 5s the patch passed +1 unit 7m 55s hadoop-common in the patch passed. +1 unit 1m 23s hadoop-hdfs-client in the patch passed. -1 unit 67m 53s hadoop-hdfs in the patch failed. +1 asflicense 0m 36s The patch does not generate ASF License warnings. 156m 15s Reason Tests Failed junit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure070   hadoop.hdfs.TestDFSRSDefault10x4StripedOutputStreamWithFailure Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-11793 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12867513/HDFS-11793.004.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc xml uname Linux 793c6f0db906 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 51b671e Default Java 1.8.0_121 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19397/artifact/patchprocess/branch-findbugs-hadoop-common-project_hadoop-common-warnings.html findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19397/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19397/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19397/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/19397/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19397/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19397/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        Sammi SammiChen added a comment -

        1. fix failed unit test. All failed unit tests passed locally.
        2. enforce super privilege on AddErasureCodingPolicies API

        Show
        Sammi SammiChen added a comment - 1. fix failed unit test. All failed unit tests passed locally. 2. enforce super privilege on AddErasureCodingPolicies API
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 25s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
        0 mvndep 1m 33s Maven dependency ordering for branch
        +1 mvninstall 14m 28s trunk passed
        +1 compile 17m 51s trunk passed
        +1 checkstyle 2m 9s trunk passed
        +1 mvnsite 2m 47s trunk passed
        +1 mvneclipse 1m 4s trunk passed
        -1 findbugs 1m 25s hadoop-common-project/hadoop-common in trunk has 19 extant Findbugs warnings.
        -1 findbugs 1m 33s hadoop-hdfs-project/hadoop-hdfs-client in trunk has 2 extant Findbugs warnings.
        -1 findbugs 1m 50s hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant Findbugs warnings.
        +1 javadoc 2m 4s trunk passed
        0 mvndep 0m 13s Maven dependency ordering for patch
        +1 mvninstall 2m 0s the patch passed
        +1 compile 13m 19s the patch passed
        +1 cc 13m 19s the patch passed
        +1 javac 13m 19s the patch passed
        -0 checkstyle 2m 7s root: The patch generated 4 new + 973 unchanged - 0 fixed = 977 total (was 973)
        +1 mvnsite 2m 44s the patch passed
        +1 mvneclipse 1m 2s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 1s The patch has no ill-formed XML file.
        -1 findbugs 1m 57s hadoop-hdfs-project/hadoop-hdfs generated 1 new + 10 unchanged - 0 fixed = 11 total (was 10)
        +1 javadoc 2m 3s the patch passed
        -1 unit 6m 59s hadoop-common in the patch failed.
        -1 unit 1m 19s hadoop-hdfs-client in the patch failed.
        -1 unit 64m 45s hadoop-hdfs in the patch failed.
        -1 asflicense 0m 36s The patch generated 1 ASF License warnings.
        150m 52s



        Reason Tests
        FindBugs module:hadoop-hdfs-project/hadoop-hdfs
          Write to static field org.apache.hadoop.hdfs.server.namenode.ErasureCodingPolicyManager.maxCellSize from instance method org.apache.hadoop.hdfs.server.namenode.ErasureCodingPolicyManager.init(Configuration) At ErasureCodingPolicyManager.java:from instance method org.apache.hadoop.hdfs.server.namenode.ErasureCodingPolicyManager.init(Configuration) At ErasureCodingPolicyManager.java:[line 122]
        Failed junit tests hadoop.fs.sftp.TestSFTPFileSystem
          hadoop.hdfs.protocol.TestErasureCodingPolicy
          hadoop.hdfs.TestDFSStripedOutputStreamWithFailure160
          hadoop.hdfs.server.balancer.TestBalancer
          hadoop.hdfs.TestDFSStripedOutputStreamWithFailure070
          hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
          hadoop.hdfs.TestDFSRSDefault10x4StripedOutputStreamWithFailure



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:14b5c93
        JIRA Issue HDFS-11793
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12867492/HDFS-11793.003.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc xml
        uname Linux d2dbee447609 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / eed7314
        Default Java 1.8.0_121
        findbugs v3.1.0-RC1
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19395/artifact/patchprocess/branch-findbugs-hadoop-common-project_hadoop-common-warnings.html
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19395/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19395/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19395/artifact/patchprocess/diff-checkstyle-root.txt
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19395/artifact/patchprocess/new-findbugs-hadoop-hdfs-project_hadoop-hdfs.html
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/19395/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/19395/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-client.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/19395/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19395/testReport/
        asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/19395/artifact/patchprocess/patch-asflicense-problems.txt
        modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19395/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 25s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files. 0 mvndep 1m 33s Maven dependency ordering for branch +1 mvninstall 14m 28s trunk passed +1 compile 17m 51s trunk passed +1 checkstyle 2m 9s trunk passed +1 mvnsite 2m 47s trunk passed +1 mvneclipse 1m 4s trunk passed -1 findbugs 1m 25s hadoop-common-project/hadoop-common in trunk has 19 extant Findbugs warnings. -1 findbugs 1m 33s hadoop-hdfs-project/hadoop-hdfs-client in trunk has 2 extant Findbugs warnings. -1 findbugs 1m 50s hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant Findbugs warnings. +1 javadoc 2m 4s trunk passed 0 mvndep 0m 13s Maven dependency ordering for patch +1 mvninstall 2m 0s the patch passed +1 compile 13m 19s the patch passed +1 cc 13m 19s the patch passed +1 javac 13m 19s the patch passed -0 checkstyle 2m 7s root: The patch generated 4 new + 973 unchanged - 0 fixed = 977 total (was 973) +1 mvnsite 2m 44s the patch passed +1 mvneclipse 1m 2s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. -1 findbugs 1m 57s hadoop-hdfs-project/hadoop-hdfs generated 1 new + 10 unchanged - 0 fixed = 11 total (was 10) +1 javadoc 2m 3s the patch passed -1 unit 6m 59s hadoop-common in the patch failed. -1 unit 1m 19s hadoop-hdfs-client in the patch failed. -1 unit 64m 45s hadoop-hdfs in the patch failed. -1 asflicense 0m 36s The patch generated 1 ASF License warnings. 150m 52s Reason Tests FindBugs module:hadoop-hdfs-project/hadoop-hdfs   Write to static field org.apache.hadoop.hdfs.server.namenode.ErasureCodingPolicyManager.maxCellSize from instance method org.apache.hadoop.hdfs.server.namenode.ErasureCodingPolicyManager.init(Configuration) At ErasureCodingPolicyManager.java:from instance method org.apache.hadoop.hdfs.server.namenode.ErasureCodingPolicyManager.init(Configuration) At ErasureCodingPolicyManager.java: [line 122] Failed junit tests hadoop.fs.sftp.TestSFTPFileSystem   hadoop.hdfs.protocol.TestErasureCodingPolicy   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure160   hadoop.hdfs.server.balancer.TestBalancer   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure070   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.TestDFSRSDefault10x4StripedOutputStreamWithFailure Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-11793 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12867492/HDFS-11793.003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc xml uname Linux d2dbee447609 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / eed7314 Default Java 1.8.0_121 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19395/artifact/patchprocess/branch-findbugs-hadoop-common-project_hadoop-common-warnings.html findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19395/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19395/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19395/artifact/patchprocess/diff-checkstyle-root.txt findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19395/artifact/patchprocess/new-findbugs-hadoop-hdfs-project_hadoop-hdfs.html unit https://builds.apache.org/job/PreCommit-HDFS-Build/19395/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/19395/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-client.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/19395/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19395/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/19395/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19395/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        Sammi SammiChen added a comment -

        1. improve test case
        2. fix check style issue
        3. fix failed unit test

        Show
        Sammi SammiChen added a comment - 1. improve test case 2. fix check style issue 3. fix failed unit test
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 22s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
        0 mvndep 1m 30s Maven dependency ordering for branch
        +1 mvninstall 15m 25s trunk passed
        +1 compile 17m 11s trunk passed
        +1 checkstyle 2m 8s trunk passed
        +1 mvnsite 2m 49s trunk passed
        +1 mvneclipse 1m 2s trunk passed
        -1 findbugs 1m 24s hadoop-common-project/hadoop-common in trunk has 19 extant Findbugs warnings.
        -1 findbugs 1m 27s hadoop-hdfs-project/hadoop-hdfs-client in trunk has 2 extant Findbugs warnings.
        -1 findbugs 1m 46s hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant Findbugs warnings.
        +1 javadoc 2m 1s trunk passed
        0 mvndep 0m 13s Maven dependency ordering for patch
        +1 mvninstall 1m 58s the patch passed
        +1 compile 14m 13s the patch passed
        +1 cc 14m 13s the patch passed
        +1 javac 14m 13s the patch passed
        -0 checkstyle 2m 6s root: The patch generated 7 new + 971 unchanged - 0 fixed = 978 total (was 971)
        +1 mvnsite 2m 45s the patch passed
        +1 mvneclipse 1m 1s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 1s The patch has no ill-formed XML file.
        -1 findbugs 1m 57s hadoop-hdfs-project/hadoop-hdfs generated 1 new + 10 unchanged - 0 fixed = 11 total (was 10)
        +1 javadoc 2m 4s the patch passed
        -1 unit 7m 30s hadoop-common in the patch failed.
        +1 unit 1m 21s hadoop-hdfs-client in the patch passed.
        -1 unit 71m 29s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 39s The patch does not generate ASF License warnings.
        159m 0s



        Reason Tests
        FindBugs module:hadoop-hdfs-project/hadoop-hdfs
          Write to static field org.apache.hadoop.hdfs.server.namenode.ErasureCodingPolicyManager.MAX_CELL_SIZE from instance method org.apache.hadoop.hdfs.server.namenode.ErasureCodingPolicyManager.init(Configuration) At ErasureCodingPolicyManager.java:from instance method org.apache.hadoop.hdfs.server.namenode.ErasureCodingPolicyManager.init(Configuration) At ErasureCodingPolicyManager.java:[line 124]
        Failed junit tests hadoop.ha.TestZKFailoverController
          hadoop.io.erasurecode.TestCodecRegistry
          hadoop.hdfs.server.namenode.TestDecommissioningStatus
          hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery
          hadoop.hdfs.TestDFSRSDefault10x4StripedOutputStreamWithFailure
        Timed out junit tests org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:14b5c93
        JIRA Issue HDFS-11793
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12867310/HDFS-11793.002.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc xml
        uname Linux 79c67c9305a9 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 2ba9903
        Default Java 1.8.0_121
        findbugs v3.1.0-RC1
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19379/artifact/patchprocess/branch-findbugs-hadoop-common-project_hadoop-common-warnings.html
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19379/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19379/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19379/artifact/patchprocess/diff-checkstyle-root.txt
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19379/artifact/patchprocess/new-findbugs-hadoop-hdfs-project_hadoop-hdfs.html
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/19379/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/19379/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19379/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19379/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 22s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 1m 30s Maven dependency ordering for branch +1 mvninstall 15m 25s trunk passed +1 compile 17m 11s trunk passed +1 checkstyle 2m 8s trunk passed +1 mvnsite 2m 49s trunk passed +1 mvneclipse 1m 2s trunk passed -1 findbugs 1m 24s hadoop-common-project/hadoop-common in trunk has 19 extant Findbugs warnings. -1 findbugs 1m 27s hadoop-hdfs-project/hadoop-hdfs-client in trunk has 2 extant Findbugs warnings. -1 findbugs 1m 46s hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant Findbugs warnings. +1 javadoc 2m 1s trunk passed 0 mvndep 0m 13s Maven dependency ordering for patch +1 mvninstall 1m 58s the patch passed +1 compile 14m 13s the patch passed +1 cc 14m 13s the patch passed +1 javac 14m 13s the patch passed -0 checkstyle 2m 6s root: The patch generated 7 new + 971 unchanged - 0 fixed = 978 total (was 971) +1 mvnsite 2m 45s the patch passed +1 mvneclipse 1m 1s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. -1 findbugs 1m 57s hadoop-hdfs-project/hadoop-hdfs generated 1 new + 10 unchanged - 0 fixed = 11 total (was 10) +1 javadoc 2m 4s the patch passed -1 unit 7m 30s hadoop-common in the patch failed. +1 unit 1m 21s hadoop-hdfs-client in the patch passed. -1 unit 71m 29s hadoop-hdfs in the patch failed. +1 asflicense 0m 39s The patch does not generate ASF License warnings. 159m 0s Reason Tests FindBugs module:hadoop-hdfs-project/hadoop-hdfs   Write to static field org.apache.hadoop.hdfs.server.namenode.ErasureCodingPolicyManager.MAX_CELL_SIZE from instance method org.apache.hadoop.hdfs.server.namenode.ErasureCodingPolicyManager.init(Configuration) At ErasureCodingPolicyManager.java:from instance method org.apache.hadoop.hdfs.server.namenode.ErasureCodingPolicyManager.init(Configuration) At ErasureCodingPolicyManager.java: [line 124] Failed junit tests hadoop.ha.TestZKFailoverController   hadoop.io.erasurecode.TestCodecRegistry   hadoop.hdfs.server.namenode.TestDecommissioningStatus   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery   hadoop.hdfs.TestDFSRSDefault10x4StripedOutputStreamWithFailure Timed out junit tests org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-11793 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12867310/HDFS-11793.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc xml uname Linux 79c67c9305a9 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 2ba9903 Default Java 1.8.0_121 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19379/artifact/patchprocess/branch-findbugs-hadoop-common-project_hadoop-common-warnings.html findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19379/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19379/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19379/artifact/patchprocess/diff-checkstyle-root.txt findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19379/artifact/patchprocess/new-findbugs-hadoop-hdfs-project_hadoop-hdfs.html unit https://builds.apache.org/job/PreCommit-HDFS-Build/19379/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/19379/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19379/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19379/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        Sammi SammiChen added a comment -

        Thanks Kai Zheng for review the patch! All the concerns are addressed in v2 patch.

        Show
        Sammi SammiChen added a comment - Thanks Kai Zheng for review the patch! All the concerns are addressed in v2 patch.
        Hide
        drankye Kai Zheng added a comment -

        Thanks Sammi for catching and working on this.

        The patch looks good to me overall. Some comments:
        1. Better to have a util method like getCoders in CodecUtil for this.

        +    if (CodecRegistry.getInstance().getCoderNames(policy.getCodecName())
        +        == null) {
        

        2. Nice to have these checks. Thanks! Could we add comments in the template file accordingly? MAX_CELL_SIZE can be larger, say 4MB.

        +    if ((policy.getCellSize() % 1024) != 0) {
        +      throw new IllegalECPolicyException("Cell size " + policy.getCellSize()
        +          + " should be 1024 aligned");
        +    }
        +
        +    if (policy.getCellSize() > ErasureCodeConstants.MAX_CELL_SIZE) {
        +      throw new IllegalECPolicyException("Cell size " + policy.getCellSize()
        +          + " should not exceed maximum " + ErasureCodeConstants.MAX_CELL_SIZE
        +          + " byte");
        +    
        

        3. Note there're test failures and check styles to fix.

        Show
        drankye Kai Zheng added a comment - Thanks Sammi for catching and working on this. The patch looks good to me overall. Some comments: 1. Better to have a util method like getCoders in CodecUtil for this. + if (CodecRegistry.getInstance().getCoderNames(policy.getCodecName()) + == null ) { 2. Nice to have these checks. Thanks! Could we add comments in the template file accordingly? MAX_CELL_SIZE can be larger, say 4MB. + if ((policy.getCellSize() % 1024) != 0) { + throw new IllegalECPolicyException( "Cell size " + policy.getCellSize() + + " should be 1024 aligned" ); + } + + if (policy.getCellSize() > ErasureCodeConstants.MAX_CELL_SIZE) { + throw new IllegalECPolicyException( "Cell size " + policy.getCellSize() + + " should not exceed maximum " + ErasureCodeConstants.MAX_CELL_SIZE + + " byte " ); + 3. Note there're test failures and check styles to fix.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 17s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
        0 mvndep 1m 42s Maven dependency ordering for branch
        +1 mvninstall 14m 40s trunk passed
        +1 compile 16m 2s trunk passed
        +1 checkstyle 2m 2s trunk passed
        +1 mvnsite 2m 50s trunk passed
        +1 mvneclipse 1m 1s trunk passed
        -1 findbugs 1m 29s hadoop-common-project/hadoop-common in trunk has 19 extant Findbugs warnings.
        -1 findbugs 1m 28s hadoop-hdfs-project/hadoop-hdfs-client in trunk has 2 extant Findbugs warnings.
        -1 findbugs 1m 46s hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant Findbugs warnings.
        +1 javadoc 2m 3s trunk passed
        0 mvndep 0m 14s Maven dependency ordering for patch
        +1 mvninstall 2m 19s the patch passed
        +1 compile 16m 8s the patch passed
        +1 cc 16m 8s the patch passed
        +1 javac 16m 8s the patch passed
        -0 checkstyle 2m 4s root: The patch generated 6 new + 562 unchanged - 0 fixed = 568 total (was 562)
        +1 mvnsite 3m 9s the patch passed
        +1 mvneclipse 0m 58s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 5m 56s the patch passed
        +1 javadoc 2m 10s the patch passed
        -1 unit 8m 24s hadoop-common in the patch failed.
        +1 unit 1m 20s hadoop-hdfs-client in the patch passed.
        -1 unit 71m 36s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 40s The patch does not generate ASF License warnings.
        161m 50s



        Reason Tests
        Failed junit tests hadoop.io.erasurecode.TestCodecRegistry
          hadoop.ha.TestZKFailoverController
          hadoop.hdfs.TestDFSRSDefault10x4StripedOutputStreamWithFailure
          hadoop.hdfs.server.namenode.ha.TestPipelinesFailover



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:14b5c93
        JIRA Issue HDFS-11793
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12867258/HDFS-11793.001.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux 9d1f7b8d88ae 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 06ffeb8
        Default Java 1.8.0_121
        findbugs v3.1.0-RC1
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19377/artifact/patchprocess/branch-findbugs-hadoop-common-project_hadoop-common-warnings.html
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19377/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19377/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19377/artifact/patchprocess/diff-checkstyle-root.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/19377/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/19377/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19377/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19377/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 1m 42s Maven dependency ordering for branch +1 mvninstall 14m 40s trunk passed +1 compile 16m 2s trunk passed +1 checkstyle 2m 2s trunk passed +1 mvnsite 2m 50s trunk passed +1 mvneclipse 1m 1s trunk passed -1 findbugs 1m 29s hadoop-common-project/hadoop-common in trunk has 19 extant Findbugs warnings. -1 findbugs 1m 28s hadoop-hdfs-project/hadoop-hdfs-client in trunk has 2 extant Findbugs warnings. -1 findbugs 1m 46s hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant Findbugs warnings. +1 javadoc 2m 3s trunk passed 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 2m 19s the patch passed +1 compile 16m 8s the patch passed +1 cc 16m 8s the patch passed +1 javac 16m 8s the patch passed -0 checkstyle 2m 4s root: The patch generated 6 new + 562 unchanged - 0 fixed = 568 total (was 562) +1 mvnsite 3m 9s the patch passed +1 mvneclipse 0m 58s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 5m 56s the patch passed +1 javadoc 2m 10s the patch passed -1 unit 8m 24s hadoop-common in the patch failed. +1 unit 1m 20s hadoop-hdfs-client in the patch passed. -1 unit 71m 36s hadoop-hdfs in the patch failed. +1 asflicense 0m 40s The patch does not generate ASF License warnings. 161m 50s Reason Tests Failed junit tests hadoop.io.erasurecode.TestCodecRegistry   hadoop.ha.TestZKFailoverController   hadoop.hdfs.TestDFSRSDefault10x4StripedOutputStreamWithFailure   hadoop.hdfs.server.namenode.ha.TestPipelinesFailover Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-11793 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12867258/HDFS-11793.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 9d1f7b8d88ae 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 06ffeb8 Default Java 1.8.0_121 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19377/artifact/patchprocess/branch-findbugs-hadoop-common-project_hadoop-common-warnings.html findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19377/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19377/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19377/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/19377/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/19377/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19377/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19377/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        Sammi SammiChen added a comment - - edited

        Initial patch.
        1. support user defined erasure coding policy enable with property dfs.namenode.ec.policies.enabled
        2. refactor part of AddErasureCodingPolicies API
        3. refactor part of CodecRegistry

        Show
        Sammi SammiChen added a comment - - edited Initial patch. 1. support user defined erasure coding policy enable with property dfs.namenode.ec.policies.enabled 2. refactor part of AddErasureCodingPolicies API 3. refactor part of CodecRegistry

          People

          • Assignee:
            Sammi SammiChen
            Reporter:
            Sammi SammiChen
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development