Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11623

Move system erasure coding policies into hadoop-hdfs-client

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-alpha2
    • Fix Version/s: 3.0.0-alpha4
    • Component/s: erasure-coding
    • Labels:
      None

      Description

      This is a precursor to HDFS-11565. We need to move the set of system defined EC policies out of the NameNode's ECPolicyManager into the hdfs-client module so it can be referenced by the client.

      1. HDFS-11623.001.patch
        47 kB
        Andrew Wang
      2. HDFS-11623.002.patch
        50 kB
        Andrew Wang
      3. HDFS-11623.003.patch
        50 kB
        Andrew Wang
      4. HDFS-11623.004.patch
        50 kB
        Andrew Wang
      5. HDFS-11623.005.patch
        50 kB
        Andrew Wang
      6. HDFS-11623.006.patch
        50 kB
        Andrew Wang

        Issue Links

          Activity

          Hide
          andrew.wang Andrew Wang added a comment -

          Patch attached, no new tests since it's an IDE assisted refactor.

          Show
          andrew.wang Andrew Wang added a comment - Patch attached, no new tests since it's an IDE assisted refactor.
          Hide
          andrew.wang Andrew Wang added a comment -

          Kai Zheng, could you review if Jenkins looks good?

          Show
          andrew.wang Andrew Wang added a comment - Kai Zheng , could you review if Jenkins looks good?
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 19 new or modified test files.
          0 mvndep 0m 23s Maven dependency ordering for branch
          +1 mvninstall 12m 52s trunk passed
          +1 compile 1m 22s trunk passed
          +1 checkstyle 0m 44s trunk passed
          +1 mvnsite 2m 9s trunk passed
          +1 mvneclipse 0m 37s trunk passed
          +1 findbugs 3m 32s trunk passed
          +1 javadoc 1m 14s trunk passed
          0 mvndep 0m 6s Maven dependency ordering for patch
          +1 mvninstall 1m 44s the patch passed
          +1 compile 1m 18s the patch passed
          +1 javac 1m 18s the patch passed
          -0 checkstyle 0m 42s hadoop-hdfs-project: The patch generated 6 new + 406 unchanged - 0 fixed = 412 total (was 406)
          +1 mvnsite 2m 2s the patch passed
          +1 mvneclipse 0m 30s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          -1 findbugs 1m 32s hadoop-hdfs-project/hadoop-hdfs-client generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
          +1 javadoc 1m 6s the patch passed
          +1 unit 0m 52s hadoop-hdfs-client in the patch passed.
          +1 unit 63m 41s hadoop-hdfs in the patch passed.
          +1 unit 3m 14s hadoop-hdfs-httpfs in the patch passed.
          -1 asflicense 0m 18s The patch generated 1 ASF License warnings.
          104m 2s



          Reason Tests
          FindBugs module:hadoop-hdfs-project/hadoop-hdfs-client
            Incorrect lazy initialization and update of static field org.apache.hadoop.hdfs.protocol.ErasureCodingPolicies.SYSTEM_POLICIES_BY_ID in org.apache.hadoop.hdfs.protocol.ErasureCodingPolicies.initIfNecessary() At ErasureCodingPolicies.java:of static field org.apache.hadoop.hdfs.protocol.ErasureCodingPolicies.SYSTEM_POLICIES_BY_ID in org.apache.hadoop.hdfs.protocol.ErasureCodingPolicies.initIfNecessary() At ErasureCodingPolicies.java:[lines 66-68]
            Public static org.apache.hadoop.hdfs.protocol.ErasureCodingPolicies.getPolicies() may expose internal representation by returning ErasureCodingPolicies.SYS_POLICIES At ErasureCodingPolicies.java:internal representation by returning ErasureCodingPolicies.SYS_POLICIES At ErasureCodingPolicies.java:[line 81]



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HDFS-11623
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12861993/HDFS-11623.001.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 857cd150b753 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 2d5c09b
          Default Java 1.8.0_121
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18977/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt
          findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/18977/artifact/patchprocess/new-findbugs-hadoop-hdfs-project_hadoop-hdfs-client.html
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18977/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/18977/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs U: hadoop-hdfs-project
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18977/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 19 new or modified test files. 0 mvndep 0m 23s Maven dependency ordering for branch +1 mvninstall 12m 52s trunk passed +1 compile 1m 22s trunk passed +1 checkstyle 0m 44s trunk passed +1 mvnsite 2m 9s trunk passed +1 mvneclipse 0m 37s trunk passed +1 findbugs 3m 32s trunk passed +1 javadoc 1m 14s trunk passed 0 mvndep 0m 6s Maven dependency ordering for patch +1 mvninstall 1m 44s the patch passed +1 compile 1m 18s the patch passed +1 javac 1m 18s the patch passed -0 checkstyle 0m 42s hadoop-hdfs-project: The patch generated 6 new + 406 unchanged - 0 fixed = 412 total (was 406) +1 mvnsite 2m 2s the patch passed +1 mvneclipse 0m 30s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. -1 findbugs 1m 32s hadoop-hdfs-project/hadoop-hdfs-client generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) +1 javadoc 1m 6s the patch passed +1 unit 0m 52s hadoop-hdfs-client in the patch passed. +1 unit 63m 41s hadoop-hdfs in the patch passed. +1 unit 3m 14s hadoop-hdfs-httpfs in the patch passed. -1 asflicense 0m 18s The patch generated 1 ASF License warnings. 104m 2s Reason Tests FindBugs module:hadoop-hdfs-project/hadoop-hdfs-client   Incorrect lazy initialization and update of static field org.apache.hadoop.hdfs.protocol.ErasureCodingPolicies.SYSTEM_POLICIES_BY_ID in org.apache.hadoop.hdfs.protocol.ErasureCodingPolicies.initIfNecessary() At ErasureCodingPolicies.java:of static field org.apache.hadoop.hdfs.protocol.ErasureCodingPolicies.SYSTEM_POLICIES_BY_ID in org.apache.hadoop.hdfs.protocol.ErasureCodingPolicies.initIfNecessary() At ErasureCodingPolicies.java: [lines 66-68]   Public static org.apache.hadoop.hdfs.protocol.ErasureCodingPolicies.getPolicies() may expose internal representation by returning ErasureCodingPolicies.SYS_POLICIES At ErasureCodingPolicies.java:internal representation by returning ErasureCodingPolicies.SYS_POLICIES At ErasureCodingPolicies.java: [line 81] Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11623 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12861993/HDFS-11623.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 857cd150b753 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 2d5c09b Default Java 1.8.0_121 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18977/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/18977/artifact/patchprocess/new-findbugs-hadoop-hdfs-project_hadoop-hdfs-client.html Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18977/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/18977/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18977/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          andrew.wang Andrew Wang added a comment -

          New patch.

          • Fix checkstyles and missing ASF header.
          • Fix findbugs. Initialize the static fields in a static block. Change getPolicies to return an unmodifiableList rather than an array, with corresponding usage updates.
          Show
          andrew.wang Andrew Wang added a comment - New patch. Fix checkstyles and missing ASF header. Fix findbugs. Initialize the static fields in a static block. Change getPolicies to return an unmodifiableList rather than an array, with corresponding usage updates.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 21s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 19 new or modified test files.
          0 mvndep 0m 25s Maven dependency ordering for branch
          +1 mvninstall 13m 2s trunk passed
          +1 compile 1m 22s trunk passed
          +1 checkstyle 0m 45s trunk passed
          +1 mvnsite 2m 10s trunk passed
          +1 mvneclipse 0m 37s trunk passed
          +1 findbugs 3m 34s trunk passed
          +1 javadoc 1m 13s trunk passed
          0 mvndep 0m 7s Maven dependency ordering for patch
          +1 mvninstall 1m 44s the patch passed
          +1 compile 1m 19s the patch passed
          +1 javac 1m 19s the patch passed
          -0 checkstyle 0m 42s hadoop-hdfs-project: The patch generated 6 new + 406 unchanged - 0 fixed = 412 total (was 406)
          +1 mvnsite 2m 3s the patch passed
          +1 mvneclipse 0m 29s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 3m 52s the patch passed
          +1 javadoc 1m 6s the patch passed
          +1 unit 1m 8s hadoop-hdfs-client in the patch passed.
          -1 unit 66m 35s hadoop-hdfs in the patch failed.
          +1 unit 3m 1s hadoop-hdfs-httpfs in the patch passed.
          +1 asflicense 0m 19s The patch does not generate ASF License warnings.
          107m 25s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
            hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HDFS-11623
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12862025/HDFS-11623.002.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 44568978bf30 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 9e0e430
          Default Java 1.8.0_121
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18981/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/18981/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18981/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs U: hadoop-hdfs-project
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18981/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 21s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 19 new or modified test files. 0 mvndep 0m 25s Maven dependency ordering for branch +1 mvninstall 13m 2s trunk passed +1 compile 1m 22s trunk passed +1 checkstyle 0m 45s trunk passed +1 mvnsite 2m 10s trunk passed +1 mvneclipse 0m 37s trunk passed +1 findbugs 3m 34s trunk passed +1 javadoc 1m 13s trunk passed 0 mvndep 0m 7s Maven dependency ordering for patch +1 mvninstall 1m 44s the patch passed +1 compile 1m 19s the patch passed +1 javac 1m 19s the patch passed -0 checkstyle 0m 42s hadoop-hdfs-project: The patch generated 6 new + 406 unchanged - 0 fixed = 412 total (was 406) +1 mvnsite 2m 3s the patch passed +1 mvneclipse 0m 29s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 52s the patch passed +1 javadoc 1m 6s the patch passed +1 unit 1m 8s hadoop-hdfs-client in the patch passed. -1 unit 66m 35s hadoop-hdfs in the patch failed. +1 unit 3m 1s hadoop-hdfs-httpfs in the patch passed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 107m 25s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11623 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12862025/HDFS-11623.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 44568978bf30 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 9e0e430 Default Java 1.8.0_121 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18981/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/18981/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18981/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18981/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          drankye Kai Zheng added a comment -

          Thanks Andrew Wang for the work!

          I haven't got a chance to read the patch yet, so far just a quick question: it looks to me the current ErasureCodingPolicyManager is still needed, and what we might want is to have something like System/BuiltInErasureCodingPolicies on client side. I mean, the manager will manage both system or built-in polices and user defined polices via the plugin mechanism. For client side codes and tests, it may be good enough to just know the built-in policies; on NN server side and related tests, it may be good to still use the manager.

          Show
          drankye Kai Zheng added a comment - Thanks Andrew Wang for the work! I haven't got a chance to read the patch yet, so far just a quick question: it looks to me the current ErasureCodingPolicyManager is still needed, and what we might want is to have something like System/BuiltInErasureCodingPolicies on client side. I mean, the manager will manage both system or built-in polices and user defined polices via the plugin mechanism. For client side codes and tests, it may be good enough to just know the built-in policies; on NN server side and related tests, it may be good to still use the manager.
          Hide
          andrew.wang Andrew Wang added a comment -

          Yep, that was precisely my plan. Already, the ECPManager is responsible for managing the set of enabled policies, which is different from the full set of system policies.

          Should I rename the newly split ErasureCodingPolicies to SystemErasureCodingPolicies for clarity?

          Show
          andrew.wang Andrew Wang added a comment - Yep, that was precisely my plan. Already, the ECPManager is responsible for managing the set of enabled policies, which is different from the full set of system policies. Should I rename the newly split ErasureCodingPolicies to SystemErasureCodingPolicies for clarity?
          Hide
          drankye Kai Zheng added a comment -

          Should I rename the newly split ErasureCodingPolicies to SystemErasureCodingPolicies for clarity?

          Yeah, that would be more accurate. Thanks!

          the ECPManager is responsible for managing the set of enabled policies, which is different from the full set of system policies.

          I got your idea, so in your view the manager is to manage the set of enabled policies (from system ones and user defined ones). In my thought, we need a central place to manage:

          • system policies, or the built-in ones;
          • user defined policies, input by an CLI command via an XML file;

          Or

          • enabled policies;
          • available policies to choose and enable;
          • maybe disabled policies? For those admins want to blacklist but can't just simply remove as already used by some data.

          It can have :

          • get a list of all policies;
          • get a policy by name or id;
          • deal with fsimage/editlog persisting user defined policies, or doing the removal;
          • maybe a method like isSystemPolicy to tell one policy is system one or not.

          Maybe server side tests prefer the manager over the new SystemErasureCodingPolicies since it will also support test of user defined policies.

          I think it's a good time to discuss what the manager should do, as quite a few issues on-going may relate to it.

          Show
          drankye Kai Zheng added a comment - Should I rename the newly split ErasureCodingPolicies to SystemErasureCodingPolicies for clarity? Yeah, that would be more accurate. Thanks! the ECPManager is responsible for managing the set of enabled policies, which is different from the full set of system policies. I got your idea, so in your view the manager is to manage the set of enabled policies (from system ones and user defined ones). In my thought, we need a central place to manage: system policies, or the built-in ones; user defined policies, input by an CLI command via an XML file; Or enabled policies; available policies to choose and enable; maybe disabled policies? For those admins want to blacklist but can't just simply remove as already used by some data. It can have : get a list of all policies; get a policy by name or id; deal with fsimage/editlog persisting user defined policies, or doing the removal; maybe a method like isSystemPolicy to tell one policy is system one or not. Maybe server side tests prefer the manager over the new SystemErasureCodingPolicies since it will also support test of user defined policies. I think it's a good time to discuss what the manager should do, as quite a few issues on-going may relate to it.
          Hide
          andrew.wang Andrew Wang added a comment -

          New patch attached renaming to SystemErasureCodingPolicies.

          The responsibilities you describe seem like they fit well within the existing ErasureCodingPolicyManager class since the set of policies is managed by the NN. Stripping out the built-in policies like this patch proposes helps modularize the code, and also clarifies the getters.

          Show
          andrew.wang Andrew Wang added a comment - New patch attached renaming to SystemErasureCodingPolicies . The responsibilities you describe seem like they fit well within the existing ErasureCodingPolicyManager class since the set of policies is managed by the NN. Stripping out the built-in policies like this patch proposes helps modularize the code, and also clarifies the getters.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 23s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 19 new or modified test files.
          0 mvndep 0m 11s Maven dependency ordering for branch
          +1 mvninstall 14m 36s trunk passed
          +1 compile 1m 37s trunk passed
          +1 checkstyle 0m 48s trunk passed
          +1 mvnsite 2m 30s trunk passed
          +1 mvneclipse 0m 44s trunk passed
          +1 findbugs 4m 18s trunk passed
          +1 javadoc 1m 19s trunk passed
          0 mvndep 0m 8s Maven dependency ordering for patch
          -1 mvninstall 0m 34s hadoop-hdfs in the patch failed.
          -1 compile 0m 56s hadoop-hdfs-project in the patch failed.
          -1 javac 0m 56s hadoop-hdfs-project in the patch failed.
          -0 checkstyle 0m 26s hadoop-hdfs-project: The patch generated 1 new + 6 unchanged - 400 fixed = 7 total (was 406)
          -1 mvnsite 0m 34s hadoop-hdfs in the patch failed.
          +1 mvneclipse 0m 33s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          -1 findbugs 0m 14s hadoop-hdfs in the patch failed.
          +1 javadoc 1m 11s the patch passed
          +1 unit 1m 12s hadoop-hdfs-client in the patch passed.
          -1 unit 0m 33s hadoop-hdfs in the patch failed.
          +1 unit 3m 29s hadoop-hdfs-httpfs in the patch passed.
          +1 asflicense 0m 18s The patch does not generate ASF License warnings.
          42m 47s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HDFS-11623
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12862378/HDFS-11623.003.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 4599aa49d192 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / a49fac5
          Default Java 1.8.0_121
          findbugs v3.0.0
          mvninstall https://builds.apache.org/job/PreCommit-HDFS-Build/19000/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt
          compile https://builds.apache.org/job/PreCommit-HDFS-Build/19000/artifact/patchprocess/patch-compile-hadoop-hdfs-project.txt
          javac https://builds.apache.org/job/PreCommit-HDFS-Build/19000/artifact/patchprocess/patch-compile-hadoop-hdfs-project.txt
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19000/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt
          mvnsite https://builds.apache.org/job/PreCommit-HDFS-Build/19000/artifact/patchprocess/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt
          findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19000/artifact/patchprocess/patch-findbugs-hadoop-hdfs-project_hadoop-hdfs.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/19000/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19000/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs U: hadoop-hdfs-project
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19000/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 23s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 19 new or modified test files. 0 mvndep 0m 11s Maven dependency ordering for branch +1 mvninstall 14m 36s trunk passed +1 compile 1m 37s trunk passed +1 checkstyle 0m 48s trunk passed +1 mvnsite 2m 30s trunk passed +1 mvneclipse 0m 44s trunk passed +1 findbugs 4m 18s trunk passed +1 javadoc 1m 19s trunk passed 0 mvndep 0m 8s Maven dependency ordering for patch -1 mvninstall 0m 34s hadoop-hdfs in the patch failed. -1 compile 0m 56s hadoop-hdfs-project in the patch failed. -1 javac 0m 56s hadoop-hdfs-project in the patch failed. -0 checkstyle 0m 26s hadoop-hdfs-project: The patch generated 1 new + 6 unchanged - 400 fixed = 7 total (was 406) -1 mvnsite 0m 34s hadoop-hdfs in the patch failed. +1 mvneclipse 0m 33s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. -1 findbugs 0m 14s hadoop-hdfs in the patch failed. +1 javadoc 1m 11s the patch passed +1 unit 1m 12s hadoop-hdfs-client in the patch passed. -1 unit 0m 33s hadoop-hdfs in the patch failed. +1 unit 3m 29s hadoop-hdfs-httpfs in the patch passed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 42m 47s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11623 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12862378/HDFS-11623.003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 4599aa49d192 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / a49fac5 Default Java 1.8.0_121 findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-HDFS-Build/19000/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt compile https://builds.apache.org/job/PreCommit-HDFS-Build/19000/artifact/patchprocess/patch-compile-hadoop-hdfs-project.txt javac https://builds.apache.org/job/PreCommit-HDFS-Build/19000/artifact/patchprocess/patch-compile-hadoop-hdfs-project.txt checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19000/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt mvnsite https://builds.apache.org/job/PreCommit-HDFS-Build/19000/artifact/patchprocess/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19000/artifact/patchprocess/patch-findbugs-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/19000/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19000/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19000/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          andrew.wang Andrew Wang added a comment -

          Forgot a closing parens...

          Show
          andrew.wang Andrew Wang added a comment - Forgot a closing parens...
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 19 new or modified test files.
          0 mvndep 0m 33s Maven dependency ordering for branch
          +1 mvninstall 14m 46s trunk passed
          +1 compile 1m 57s trunk passed
          +1 checkstyle 0m 59s trunk passed
          +1 mvnsite 2m 19s trunk passed
          +1 mvneclipse 0m 37s trunk passed
          +1 findbugs 3m 47s trunk passed
          +1 javadoc 1m 16s trunk passed
          0 mvndep 0m 7s Maven dependency ordering for patch
          -1 mvninstall 0m 31s hadoop-hdfs-client in the patch failed.
          -1 mvninstall 0m 25s hadoop-hdfs in the patch failed.
          -1 mvninstall 0m 27s hadoop-hdfs-httpfs in the patch failed.
          +1 compile 1m 23s the patch passed
          +1 javac 1m 23s the patch passed
          -0 checkstyle 0m 41s hadoop-hdfs-project: The patch generated 2 new + 405 unchanged - 1 fixed = 407 total (was 406)
          -1 mvnsite 0m 27s hadoop-hdfs in the patch failed.
          +1 mvneclipse 0m 30s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          -1 findbugs 0m 24s hadoop-hdfs in the patch failed.
          -1 javadoc 0m 37s hadoop-hdfs-project_hadoop-hdfs generated 4 new + 9 unchanged - 0 fixed = 13 total (was 9)
          +1 unit 1m 8s hadoop-hdfs-client in the patch passed.
          -1 unit 0m 24s hadoop-hdfs in the patch failed.
          +1 unit 3m 14s hadoop-hdfs-httpfs in the patch passed.
          +1 asflicense 0m 17s The patch does not generate ASF License warnings.
          42m 21s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HDFS-11623
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12862394/HDFS-11623.004.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 6f2f29341612 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 0eacd4c
          Default Java 1.8.0_121
          findbugs v3.0.0
          mvninstall https://builds.apache.org/job/PreCommit-HDFS-Build/19001/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs-client.txt
          mvninstall https://builds.apache.org/job/PreCommit-HDFS-Build/19001/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt
          mvninstall https://builds.apache.org/job/PreCommit-HDFS-Build/19001/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs-httpfs.txt
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19001/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt
          mvnsite https://builds.apache.org/job/PreCommit-HDFS-Build/19001/artifact/patchprocess/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt
          findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19001/artifact/patchprocess/patch-findbugs-hadoop-hdfs-project_hadoop-hdfs.txt
          javadoc https://builds.apache.org/job/PreCommit-HDFS-Build/19001/artifact/patchprocess/diff-javadoc-javadoc-hadoop-hdfs-project_hadoop-hdfs.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/19001/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19001/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs U: hadoop-hdfs-project
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19001/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 19 new or modified test files. 0 mvndep 0m 33s Maven dependency ordering for branch +1 mvninstall 14m 46s trunk passed +1 compile 1m 57s trunk passed +1 checkstyle 0m 59s trunk passed +1 mvnsite 2m 19s trunk passed +1 mvneclipse 0m 37s trunk passed +1 findbugs 3m 47s trunk passed +1 javadoc 1m 16s trunk passed 0 mvndep 0m 7s Maven dependency ordering for patch -1 mvninstall 0m 31s hadoop-hdfs-client in the patch failed. -1 mvninstall 0m 25s hadoop-hdfs in the patch failed. -1 mvninstall 0m 27s hadoop-hdfs-httpfs in the patch failed. +1 compile 1m 23s the patch passed +1 javac 1m 23s the patch passed -0 checkstyle 0m 41s hadoop-hdfs-project: The patch generated 2 new + 405 unchanged - 1 fixed = 407 total (was 406) -1 mvnsite 0m 27s hadoop-hdfs in the patch failed. +1 mvneclipse 0m 30s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. -1 findbugs 0m 24s hadoop-hdfs in the patch failed. -1 javadoc 0m 37s hadoop-hdfs-project_hadoop-hdfs generated 4 new + 9 unchanged - 0 fixed = 13 total (was 9) +1 unit 1m 8s hadoop-hdfs-client in the patch passed. -1 unit 0m 24s hadoop-hdfs in the patch failed. +1 unit 3m 14s hadoop-hdfs-httpfs in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 42m 21s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11623 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12862394/HDFS-11623.004.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 6f2f29341612 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 0eacd4c Default Java 1.8.0_121 findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-HDFS-Build/19001/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs-client.txt mvninstall https://builds.apache.org/job/PreCommit-HDFS-Build/19001/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt mvninstall https://builds.apache.org/job/PreCommit-HDFS-Build/19001/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs-httpfs.txt checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19001/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt mvnsite https://builds.apache.org/job/PreCommit-HDFS-Build/19001/artifact/patchprocess/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19001/artifact/patchprocess/patch-findbugs-hadoop-hdfs-project_hadoop-hdfs.txt javadoc https://builds.apache.org/job/PreCommit-HDFS-Build/19001/artifact/patchprocess/diff-javadoc-javadoc-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/19001/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19001/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19001/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          drankye Kai Zheng added a comment -

          Hi Andrew,

          Thanks for your update! I understand your point.

          For changes like below, do we need to change it back when user defined policies are supported? It's OK for me to have changes like this, and we can later get back some of the removed methods like getPolicyByID for ErasureCodingPolicyManager when working on plugin-ed policies.

          @@ -302,7 +303,7 @@ private static ErasureCodingPolicy getErasureCodingPolicyForPath(
                   if (inode.isFile()) {
                     byte id = inode.asFile().getErasureCodingPolicyID();
                     return id < 0 ? null :
          -              ErasureCodingPolicyManager.getPolicyByID(id);
          +              SystemErasureCodingPolicies.getByID(id);
                   }
          
          Show
          drankye Kai Zheng added a comment - Hi Andrew, Thanks for your update! I understand your point. For changes like below, do we need to change it back when user defined policies are supported? It's OK for me to have changes like this, and we can later get back some of the removed methods like getPolicyByID for ErasureCodingPolicyManager when working on plugin-ed policies. @@ -302,7 +303,7 @@ private static ErasureCodingPolicy getErasureCodingPolicyForPath( if (inode.isFile()) { byte id = inode.asFile().getErasureCodingPolicyID(); return id < 0 ? null : - ErasureCodingPolicyManager.getPolicyByID(id); + SystemErasureCodingPolicies.getByID(id); }
          Hide
          andrew.wang Andrew Wang added a comment -

          Aha, now I understand better your concern. I did an IDE refactor to move these methods to SystemErasureCodingPolicies. You're right that with user-defined policies, some of these changed places will change back to call an ECPManager getter that can return both system policies and user-defined policies.

          As an aside, I checked all the getPolicy usage earlier when implementing enabling/disabling of EC system policies. We probably should do that again once user-defined policies are ready.

          Show
          andrew.wang Andrew Wang added a comment - Aha, now I understand better your concern. I did an IDE refactor to move these methods to SystemErasureCodingPolicies. You're right that with user-defined policies, some of these changed places will change back to call an ECPManager getter that can return both system policies and user-defined policies. As an aside, I checked all the getPolicy usage earlier when implementing enabling/disabling of EC system policies. We probably should do that again once user-defined policies are ready.
          Hide
          drankye Kai Zheng added a comment -

          Thanks for your clarifying. The work looks good to me overall. I suggest Wei-Chiu Chuang also take a look since it's related to HDFS-11565?

          Show
          drankye Kai Zheng added a comment - Thanks for your clarifying. The work looks good to me overall. I suggest Wei-Chiu Chuang also take a look since it's related to HDFS-11565 ?
          Hide
          jojochuang Wei-Chiu Chuang added a comment - - edited

          Overall looks good to me, so I am +1. I just want to point out a few things for discussion.

          1. After this change, ErasureCodingPolicyManager becomes a little redundancy. It is only useful for DistributedFileSystem#setErasureCodingPolicy() and DistributedFileSystem#getErasureCodingPolicies().
          2. Also, the concept of enabled policies is a little foreign to me. If the system has a file with ec policy A, but later the system disables the ec policy A, that file is still fine. It's just that no more new files can be created with ec policy A.
          3. My editor told me the following code:
              private static final List<ErasureCodingPolicy> SYS_POLICIES =
                  Collections.unmodifiableList(Arrays.asList(
                      new ErasureCodingPolicy[]{
                          SYS_POLICY1, SYS_POLICY2, SYS_POLICY3, SYS_POLICY4,
                          SYS_POLICY5}));
            

            can be simplified to

              private static final List<ErasureCodingPolicy> SYS_POLICIES =
                  Collections.unmodifiableList(Arrays.asList(
                      SYS_POLICY1, SYS_POLICY2, SYS_POLICY3, SYS_POLICY4, SYS_POLICY5));
            

          By the way, I found a critical issue that I will post later. HDFS-11633. This is unrelated to this patch but I found it while reviewing it.

          Show
          jojochuang Wei-Chiu Chuang added a comment - - edited Overall looks good to me, so I am +1. I just want to point out a few things for discussion. After this change, ErasureCodingPolicyManager becomes a little redundancy. It is only useful for DistributedFileSystem#setErasureCodingPolicy() and DistributedFileSystem#getErasureCodingPolicies() . Also, the concept of enabled policies is a little foreign to me. If the system has a file with ec policy A, but later the system disables the ec policy A, that file is still fine. It's just that no more new files can be created with ec policy A. My editor told me the following code: private static final List<ErasureCodingPolicy> SYS_POLICIES = Collections.unmodifiableList(Arrays.asList( new ErasureCodingPolicy[]{ SYS_POLICY1, SYS_POLICY2, SYS_POLICY3, SYS_POLICY4, SYS_POLICY5})); can be simplified to private static final List<ErasureCodingPolicy> SYS_POLICIES = Collections.unmodifiableList(Arrays.asList( SYS_POLICY1, SYS_POLICY2, SYS_POLICY3, SYS_POLICY4, SYS_POLICY5)); By the way, I found a critical issue that I will post later. HDFS-11633 . This is unrelated to this patch but I found it while reviewing it.
          Hide
          andrew.wang Andrew Wang added a comment -

          Thanks for reviewing Wei-chiu! New patch to simplify the array as noted, and also to hopefully get a working precommit. Not sure why the build failed like that, it builds for me locally.

          Enabled policies work as you describe. The idea is that admins can restrict the set of policies used by the cluster. It doesn't affect already written files since they should still be readable. It doesn't affect already set policies to reduce user surprise, though we chould reconsider this and instead treat it similarly to min-replication. Thoughts?

          Agree that ECPManager looks quite simple now, but it'll be fleshed by the pluggable policy work at HDFS-7337. Kai sketched out the new responsibilities above.

          Show
          andrew.wang Andrew Wang added a comment - Thanks for reviewing Wei-chiu! New patch to simplify the array as noted, and also to hopefully get a working precommit. Not sure why the build failed like that, it builds for me locally. Enabled policies work as you describe. The idea is that admins can restrict the set of policies used by the cluster. It doesn't affect already written files since they should still be readable. It doesn't affect already set policies to reduce user surprise, though we chould reconsider this and instead treat it similarly to min-replication. Thoughts? Agree that ECPManager looks quite simple now, but it'll be fleshed by the pluggable policy work at HDFS-7337 . Kai sketched out the new responsibilities above.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Cool. That makes sense to me. Thanks!

          Show
          jojochuang Wei-Chiu Chuang added a comment - Cool. That makes sense to me. Thanks!
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 19 new or modified test files.
          0 mvndep 0m 31s Maven dependency ordering for branch
          +1 mvninstall 14m 16s trunk passed
          +1 compile 1m 23s trunk passed
          +1 checkstyle 0m 45s trunk passed
          +1 mvnsite 2m 13s trunk passed
          +1 mvneclipse 0m 39s trunk passed
          +1 findbugs 3m 45s trunk passed
          +1 javadoc 1m 14s trunk passed
          0 mvndep 0m 7s Maven dependency ordering for patch
          +1 mvninstall 1m 46s the patch passed
          +1 compile 1m 20s the patch passed
          +1 javac 1m 20s the patch passed
          -0 checkstyle 0m 43s hadoop-hdfs-project: The patch generated 2 new + 405 unchanged - 1 fixed = 407 total (was 406)
          +1 mvnsite 2m 3s the patch passed
          +1 mvneclipse 0m 30s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 3m 52s the patch passed
          +1 javadoc 1m 7s the patch passed
          +1 unit 1m 9s hadoop-hdfs-client in the patch passed.
          -1 unit 63m 49s hadoop-hdfs in the patch failed.
          +1 unit 3m 14s hadoop-hdfs-httpfs in the patch passed.
          +1 asflicense 0m 19s The patch does not generate ASF License warnings.
          106m 34s



          Reason Tests
          Failed junit tests hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
            hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HDFS-11623
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12862515/HDFS-11623.005.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux c0b3ead79699 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 96cbb4f
          Default Java 1.8.0_121
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19012/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/19012/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19012/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs U: hadoop-hdfs-project
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19012/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 19 new or modified test files. 0 mvndep 0m 31s Maven dependency ordering for branch +1 mvninstall 14m 16s trunk passed +1 compile 1m 23s trunk passed +1 checkstyle 0m 45s trunk passed +1 mvnsite 2m 13s trunk passed +1 mvneclipse 0m 39s trunk passed +1 findbugs 3m 45s trunk passed +1 javadoc 1m 14s trunk passed 0 mvndep 0m 7s Maven dependency ordering for patch +1 mvninstall 1m 46s the patch passed +1 compile 1m 20s the patch passed +1 javac 1m 20s the patch passed -0 checkstyle 0m 43s hadoop-hdfs-project: The patch generated 2 new + 405 unchanged - 1 fixed = 407 total (was 406) +1 mvnsite 2m 3s the patch passed +1 mvneclipse 0m 30s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 52s the patch passed +1 javadoc 1m 7s the patch passed +1 unit 1m 9s hadoop-hdfs-client in the patch passed. -1 unit 63m 49s hadoop-hdfs in the patch failed. +1 unit 3m 14s hadoop-hdfs-httpfs in the patch passed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 106m 34s Reason Tests Failed junit tests hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11623 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12862515/HDFS-11623.005.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c0b3ead79699 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 96cbb4f Default Java 1.8.0_121 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19012/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/19012/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19012/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19012/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          andrew.wang Andrew Wang added a comment -

          Patch attached to fix TestOIV failure, have to use XOR for this.

          Show
          andrew.wang Andrew Wang added a comment - Patch attached to fix TestOIV failure, have to use XOR for this.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 19 new or modified test files.
          0 mvndep 0m 32s Maven dependency ordering for branch
          +1 mvninstall 13m 31s trunk passed
          +1 compile 1m 34s trunk passed
          +1 checkstyle 0m 45s trunk passed
          +1 mvnsite 2m 13s trunk passed
          +1 mvneclipse 0m 38s trunk passed
          +1 findbugs 3m 38s trunk passed
          +1 javadoc 1m 14s trunk passed
          0 mvndep 0m 7s Maven dependency ordering for patch
          +1 mvninstall 1m 43s the patch passed
          +1 compile 1m 22s the patch passed
          +1 javac 1m 22s the patch passed
          -0 checkstyle 0m 43s hadoop-hdfs-project: The patch generated 1 new + 405 unchanged - 1 fixed = 406 total (was 406)
          +1 mvnsite 2m 12s the patch passed
          +1 mvneclipse 0m 31s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 4m 0s the patch passed
          +1 javadoc 1m 17s the patch passed
          +1 unit 1m 18s hadoop-hdfs-client in the patch passed.
          -1 unit 69m 57s hadoop-hdfs in the patch failed.
          +1 unit 3m 15s hadoop-hdfs-httpfs in the patch passed.
          +1 asflicense 0m 18s The patch does not generate ASF License warnings.
          112m 42s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
            hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl
            hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HDFS-11623
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12862535/HDFS-11623.006.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux f8b5febc0026 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / d298f73
          Default Java 1.8.0_121
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19017/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/19017/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19017/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs U: hadoop-hdfs-project
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19017/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 19 new or modified test files. 0 mvndep 0m 32s Maven dependency ordering for branch +1 mvninstall 13m 31s trunk passed +1 compile 1m 34s trunk passed +1 checkstyle 0m 45s trunk passed +1 mvnsite 2m 13s trunk passed +1 mvneclipse 0m 38s trunk passed +1 findbugs 3m 38s trunk passed +1 javadoc 1m 14s trunk passed 0 mvndep 0m 7s Maven dependency ordering for patch +1 mvninstall 1m 43s the patch passed +1 compile 1m 22s the patch passed +1 javac 1m 22s the patch passed -0 checkstyle 0m 43s hadoop-hdfs-project: The patch generated 1 new + 405 unchanged - 1 fixed = 406 total (was 406) +1 mvnsite 2m 12s the patch passed +1 mvneclipse 0m 31s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 4m 0s the patch passed +1 javadoc 1m 17s the patch passed +1 unit 1m 18s hadoop-hdfs-client in the patch passed. -1 unit 69m 57s hadoop-hdfs in the patch failed. +1 unit 3m 15s hadoop-hdfs-httpfs in the patch passed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 112m 42s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11623 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12862535/HDFS-11623.006.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux f8b5febc0026 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / d298f73 Default Java 1.8.0_121 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19017/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/19017/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19017/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19017/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          andrew.wang Andrew Wang added a comment -

          Filed HADOOP-14293 for the TestFsDatasetImpl failure, others are unrelated and passed locally.

          Going to commit this shortly based on Wei-chiu's earlier +1.

          Show
          andrew.wang Andrew Wang added a comment - Filed HADOOP-14293 for the TestFsDatasetImpl failure, others are unrelated and passed locally. Going to commit this shortly based on Wei-chiu's earlier +1.
          Hide
          andrew.wang Andrew Wang added a comment -

          Committed to trunk, thanks Kai and Wei-chiu for reviewing!

          Show
          andrew.wang Andrew Wang added a comment - Committed to trunk, thanks Kai and Wei-chiu for reviewing!
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11552 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11552/)
          HDFS-11623. Move system erasure coding policies into hadoop-hdfs-client. (wang: rev e8bdad7385ab63a122c1e8e8a6a73e0f1100e80b)

          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestErasureCodingPolicies.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImage.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormatPBINode.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/StripedFileTestUtil.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSRSDefault10x4StripedInputStream.java
          • (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/SystemErasureCodingPolicies.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestStripedINodeFile.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/test/TestHdfsHelper.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockRecovery.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSRSDefault10x4StripedOutputStream.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewer.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsConstants.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestINodeFile.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSXORStripedInputStream.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/util/TestStripedBlockUtil.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestUnsetAndChangeDirectoryEcPolicy.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSXORStripedOutputStreamWithFailure.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ErasureCodingPolicyManager.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSRSDefault10x4StripedOutputStreamWithFailure.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSXORStripedOutputStream.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirErasureCodingOp.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEnabledECPolicies.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11552 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11552/ ) HDFS-11623 . Move system erasure coding policies into hadoop-hdfs-client. (wang: rev e8bdad7385ab63a122c1e8e8a6a73e0f1100e80b) (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestErasureCodingPolicies.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImage.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormatPBINode.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/StripedFileTestUtil.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSRSDefault10x4StripedInputStream.java (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/SystemErasureCodingPolicies.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestStripedINodeFile.java (edit) hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/test/TestHdfsHelper.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockRecovery.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSRSDefault10x4StripedOutputStream.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewer.java (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsConstants.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestINodeFile.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSXORStripedInputStream.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/util/TestStripedBlockUtil.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestUnsetAndChangeDirectoryEcPolicy.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSXORStripedOutputStreamWithFailure.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ErasureCodingPolicyManager.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSRSDefault10x4StripedOutputStreamWithFailure.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSXORStripedOutputStream.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirErasureCodingOp.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEnabledECPolicies.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11591 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11591/)
          HDFS-11623. Move system erasure coding policies into hadoop-hdfs-client. (wang: rev e8bdad7385ab63a122c1e8e8a6a73e0f1100e80b)

          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockRecovery.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestUnsetAndChangeDirectoryEcPolicy.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/util/TestStripedBlockUtil.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewer.java
          • (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/SystemErasureCodingPolicies.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormatPBINode.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSXORStripedOutputStreamWithFailure.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSXORStripedOutputStream.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsConstants.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestErasureCodingPolicies.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImage.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSRSDefault10x4StripedOutputStream.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/StripedFileTestUtil.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSXORStripedInputStream.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSRSDefault10x4StripedOutputStreamWithFailure.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirErasureCodingOp.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSRSDefault10x4StripedInputStream.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestStripedINodeFile.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEnabledECPolicies.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestINodeFile.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/test/TestHdfsHelper.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ErasureCodingPolicyManager.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11591 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11591/ ) HDFS-11623 . Move system erasure coding policies into hadoop-hdfs-client. (wang: rev e8bdad7385ab63a122c1e8e8a6a73e0f1100e80b) (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockRecovery.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestUnsetAndChangeDirectoryEcPolicy.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/util/TestStripedBlockUtil.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewer.java (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/SystemErasureCodingPolicies.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormatPBINode.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSXORStripedOutputStreamWithFailure.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSXORStripedOutputStream.java (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsConstants.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestErasureCodingPolicies.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImage.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSRSDefault10x4StripedOutputStream.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/StripedFileTestUtil.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSXORStripedInputStream.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSRSDefault10x4StripedOutputStreamWithFailure.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirErasureCodingOp.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSRSDefault10x4StripedInputStream.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestStripedINodeFile.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEnabledECPolicies.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestINodeFile.java (edit) hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/test/TestHdfsHelper.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ErasureCodingPolicyManager.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java

            People

            • Assignee:
              andrew.wang Andrew Wang
              Reporter:
              andrew.wang Andrew Wang
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development