Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11428

Change setErasureCodingPolicy to take a required string EC policy name

    Details

    • Target Version/s:
    • Hadoop Flags:
      Incompatible change
    • Release Note:
      {{HdfsAdmin#setErasureCodingPolicy}} now takes a String {{ecPolicyName}} rather than an ErasureCodingPolicy object. The corresponding RPC's wire format has also been modified.

      Description

      The current setErasureCodingPolicy API takes an optional ECPolicy. This makes calling the API harder for clients, since they need to turn a specified name into a policy, and the set of available EC policies is only available on the NN.

      You can see this awkwardness in the current EC cli set command: it first fetches the list of EC policies, looks for the one specified by the user, then calls set. This means we need to issue two RPCs for every set (inefficient), and we need to do validation on the NN side anyway (extraneous work).

      Since we're phasing out the system default EC policy, it also makes sense to make the policy a required parameter.

      1. HDFS-11428.001.patch
        36 kB
        Andrew Wang
      2. HDFS-11428.002.patch
        36 kB
        Andrew Wang
      3. HDFS-11428.003.patch
        65 kB
        Andrew Wang
      4. HDFS-11428.004.patch
        65 kB
        Andrew Wang
      5. HDFS-11428.005.patch
        66 kB
        Andrew Wang

        Activity

        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11326 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11326/)
        HDFS-11428. Change setErasureCodingPolicy to take a required string EC (rakeshr: rev 82ef9accafe7318278efb169678e17065e082c8e)

        • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsAdmin.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReconstructStripedBlocksWithRackAwareness.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStream.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDecommissionWithStriped.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestAddStripedBlockInFBR.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/ClientProtocol.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestSafeModeWithStripedFile.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientNamenodeProtocolTranslatorPB.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testErasureCodingConf.xml
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImage.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeErasureCodingMetrics.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestLeaseRecoveryStriped.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSEditLogLoader.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewer.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockTokenWithDFSStriped.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestErasureCodingPolicies.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewerWithStripedBlocks.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStreamWithFailure.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReadStripedFileWithDecoding.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestWriteReadStripedFile.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestStripedINodeFile.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestAddOverReplicatedStripedBlocks.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestAddStripedBlocks.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestSequentialBlockGroupId.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestQuotaWithStripedBlocks.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedInputStream.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestErasureCodingPolicyWithSnapshot.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/ECAdmin.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestUnsetAndChangeDirectoryEcPolicy.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientNamenodeProtocolServerSideTranslatorPB.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestReconstructStripedBlocks.java
        • (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/TestDFSIO.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/erasurecoding.proto
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileStatusWithECPolicy.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirErasureCodingOp.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReadStripedFileWithMissingBlocks.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/mover/TestMover.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestWriteStripedFileWithFailure.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/ErasureCodeBenchmarkThroughput.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileChecksum.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReconstructStripedFile.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11326 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11326/ ) HDFS-11428 . Change setErasureCodingPolicy to take a required string EC (rakeshr: rev 82ef9accafe7318278efb169678e17065e082c8e) (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsAdmin.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReconstructStripedBlocksWithRackAwareness.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStream.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDecommissionWithStriped.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestAddStripedBlockInFBR.java (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/ClientProtocol.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestSafeModeWithStripedFile.java (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientNamenodeProtocolTranslatorPB.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testErasureCodingConf.xml (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImage.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeErasureCodingMetrics.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestLeaseRecoveryStriped.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSEditLogLoader.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewer.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockTokenWithDFSStriped.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestErasureCodingPolicies.java (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewerWithStripedBlocks.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStreamWithFailure.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReadStripedFileWithDecoding.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestWriteReadStripedFile.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestStripedINodeFile.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestAddOverReplicatedStripedBlocks.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestAddStripedBlocks.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestSequentialBlockGroupId.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestQuotaWithStripedBlocks.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedInputStream.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestErasureCodingPolicyWithSnapshot.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/ECAdmin.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestUnsetAndChangeDirectoryEcPolicy.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientNamenodeProtocolServerSideTranslatorPB.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestReconstructStripedBlocks.java (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/TestDFSIO.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/erasurecoding.proto (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileStatusWithECPolicy.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirErasureCodingOp.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReadStripedFileWithMissingBlocks.java (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/mover/TestMover.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestWriteStripedFileWithFailure.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/ErasureCodeBenchmarkThroughput.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileChecksum.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestReconstructStripedFile.java
        Hide
        rakeshr Rakesh R added a comment -

        Committed to trunk

        Show
        rakeshr Rakesh R added a comment - Committed to trunk
        Hide
        rakeshr Rakesh R added a comment -

        Thanks Andrew Wang for the contribution.
        Thanks Wei-Chiu Chuang for the reviews.

        It looks like test cases failures are not related to the patch. I will commit the patch shortly.

        Show
        rakeshr Rakesh R added a comment - Thanks Andrew Wang for the contribution. Thanks Wei-Chiu Chuang for the reviews. It looks like test cases failures are not related to the patch. I will commit the patch shortly.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 47s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 38 new or modified test files.
        0 mvndep 0m 24s Maven dependency ordering for branch
        +1 mvninstall 19m 27s trunk passed
        +1 compile 22m 19s trunk passed
        +1 checkstyle 3m 28s trunk passed
        +1 mvnsite 4m 8s trunk passed
        +1 mvneclipse 1m 38s trunk passed
        +1 findbugs 6m 17s trunk passed
        +1 javadoc 2m 35s trunk passed
        0 mvndep 0m 25s Maven dependency ordering for patch
        +1 mvninstall 3m 6s the patch passed
        +1 compile 16m 1s the patch passed
        +1 cc 16m 1s the patch passed
        +1 javac 16m 1s the patch passed
        +1 checkstyle 2m 27s the patch passed
        +1 mvnsite 3m 5s the patch passed
        +1 mvneclipse 1m 19s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 2s The patch has no ill-formed XML file.
        +1 findbugs 5m 22s the patch passed
        +1 javadoc 2m 2s the patch passed
        +1 unit 1m 22s hadoop-hdfs-client in the patch passed.
        -1 unit 102m 11s hadoop-hdfs in the patch failed.
        -1 unit 146m 25s hadoop-mapreduce-client-jobclient in the patch failed.
        +1 asflicense 1m 48s The patch does not generate ASF License warnings.
        349m 52s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
          hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery
          hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
          hadoop.hdfs.server.datanode.TestDataNodeMXBean
          hadoop.hdfs.server.namenode.TestNamenodeCapacityReport
          hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration
          hadoop.hdfs.server.datanode.TestDirectoryScanner
          hadoop.hdfs.server.datanode.TestDataNodeUUID
          hadoop.mapred.TestMRTimelineEventHandling
        Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2
          org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean
          org.apache.hadoop.hdfs.server.namenode.TestParallelImageWrite
          org.apache.hadoop.hdfs.server.namenode.TestAclConfigFlag
          org.apache.hadoop.hdfs.server.namenode.TestDeleteRace
          org.apache.hadoop.hdfs.server.namenode.TestProtectedDirectories
          org.apache.hadoop.mapreduce.v2.TestMRJobs



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11428
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12855245/HDFS-11428.005.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc xml
        uname Linux 35f095e28475 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 235203d
        Default Java 1.8.0_121
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/18474/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/18474/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18474/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: .
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18474/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 47s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 38 new or modified test files. 0 mvndep 0m 24s Maven dependency ordering for branch +1 mvninstall 19m 27s trunk passed +1 compile 22m 19s trunk passed +1 checkstyle 3m 28s trunk passed +1 mvnsite 4m 8s trunk passed +1 mvneclipse 1m 38s trunk passed +1 findbugs 6m 17s trunk passed +1 javadoc 2m 35s trunk passed 0 mvndep 0m 25s Maven dependency ordering for patch +1 mvninstall 3m 6s the patch passed +1 compile 16m 1s the patch passed +1 cc 16m 1s the patch passed +1 javac 16m 1s the patch passed +1 checkstyle 2m 27s the patch passed +1 mvnsite 3m 5s the patch passed +1 mvneclipse 1m 19s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 findbugs 5m 22s the patch passed +1 javadoc 2m 2s the patch passed +1 unit 1m 22s hadoop-hdfs-client in the patch passed. -1 unit 102m 11s hadoop-hdfs in the patch failed. -1 unit 146m 25s hadoop-mapreduce-client-jobclient in the patch failed. +1 asflicense 1m 48s The patch does not generate ASF License warnings. 349m 52s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.server.datanode.TestDataNodeMXBean   hadoop.hdfs.server.namenode.TestNamenodeCapacityReport   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration   hadoop.hdfs.server.datanode.TestDirectoryScanner   hadoop.hdfs.server.datanode.TestDataNodeUUID   hadoop.mapred.TestMRTimelineEventHandling Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2   org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean   org.apache.hadoop.hdfs.server.namenode.TestParallelImageWrite   org.apache.hadoop.hdfs.server.namenode.TestAclConfigFlag   org.apache.hadoop.hdfs.server.namenode.TestDeleteRace   org.apache.hadoop.hdfs.server.namenode.TestProtectedDirectories   org.apache.hadoop.mapreduce.v2.TestMRJobs Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11428 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12855245/HDFS-11428.005.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc xml uname Linux 35f095e28475 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 235203d Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/18474/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/18474/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18474/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18474/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        andrew.wang Andrew Wang added a comment -

        New patch attached to fix rebase issues, thanks for reviewing!

        Show
        andrew.wang Andrew Wang added a comment - New patch attached to fix rebase issues, thanks for reviewing!
        Hide
        rakeshr Rakesh R added a comment -

        Thank you Andrew Wang, +1 latest patch looks good to me.

        It seems there are few conflicts while applying the patch. Could you please rebase the current patch on latest trunk code. I will commit it soon.

        Show
        rakeshr Rakesh R added a comment - Thank you Andrew Wang , +1 latest patch looks good to me. It seems there are few conflicts while applying the patch. Could you please rebase the current patch on latest trunk code. I will commit it soon.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        -1 patch 0m 9s HDFS-11428 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



        Subsystem Report/Notes
        JIRA Issue HDFS-11428
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12855002/HDFS-11428.004.patch
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18466/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 9s HDFS-11428 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Issue HDFS-11428 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12855002/HDFS-11428.004.patch Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18466/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Sorry that was not clear. +1 from me.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Sorry that was not clear. +1 from me.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Andrew Wang yep that's good for me. Thanks a lot for the work.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Andrew Wang yep that's good for me. Thanks a lot for the work.
        Hide
        andrew.wang Andrew Wang added a comment -

        Thanks for the review Rakesh, new patch attached that should address your comments.

        Wei-chiu, is this okay with you? I'd like to address any locking concerns in a separate patch, to keep this one focused on the change at hand.

        Show
        andrew.wang Andrew Wang added a comment - Thanks for the review Rakesh, new patch attached that should address your comments. Wei-chiu, is this okay with you? I'd like to address any locking concerns in a separate patch, to keep this one focused on the change at hand.
        Hide
        rakeshr Rakesh R added a comment -

        Thanks Andrew Wang for the useful improvement. Overall patch looks good to me. I've few suggestions,

        1. Its good to update javadoc with @throws IllegalArgumentException if the given ecPolicyName is invalid.
        2. Minor suggestion to validate null ecPolicyName, otw it will throw NPE.
          DistributedFileSystem#setErasureCodingPolicy()
          
              if (ecPolicyName == null) {
                throw new IOException("Invalid erasure coding policy name");
              }
          
        Show
        rakeshr Rakesh R added a comment - Thanks Andrew Wang for the useful improvement. Overall patch looks good to me. I've few suggestions, Its good to update javadoc with @throws IllegalArgumentException if the given ecPolicyName is invalid . Minor suggestion to validate null ecPolicyName , otw it will throw NPE. DistributedFileSystem#setErasureCodingPolicy() if (ecPolicyName == null ) { throw new IOException( "Invalid erasure coding policy name" ); }
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        FYI, there's a FSDirErasureCodingOp#getErasureCodingPolicies, which only requires fsn read lock. IMO, acquiring fsd write lock is not necessary. To ensure consistency of ec policies, the synchronization should be imposed by ErasureCodingPolicyManager access methods (e.g. ErasureCodingPolicyManager#getPolicies, ErasureCodingPolicyManager#getPolicyByName)

        Show
        jojochuang Wei-Chiu Chuang added a comment - FYI, there's a FSDirErasureCodingOp#getErasureCodingPolicies, which only requires fsn read lock. IMO, acquiring fsd write lock is not necessary. To ensure consistency of ec policies, the synchronization should be imposed by ErasureCodingPolicyManager access methods (e.g. ErasureCodingPolicyManager#getPolicies , ErasureCodingPolicyManager#getPolicyByName )
        Hide
        andrew.wang Andrew Wang added a comment -

        Good question, it's read-only except for startup and shutdown, though we might eventually be loading custom policies from NN metadata or configuration. Not clear how that's going to work yet.

        Show
        andrew.wang Andrew Wang added a comment - Good question, it's read-only except for startup and shutdown, though we might eventually be loading custom policies from NN metadata or configuration. Not clear how that's going to work yet.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Just out of curiosity, are we supporting adding more ec policies at runtime? Currently pretty much all access to ec policies are read-only.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Just out of curiosity, are we supporting adding more ec policies at runtime? Currently pretty much all access to ec policies are read-only.
        Hide
        andrew.wang Andrew Wang added a comment -

        Thanks for reviewing Wei-chiu! The list of active policies is stored in a map, and the fsd lock protects it from concurrent access. I think we should probably keep it within the lock? A single treemap lookup shouldn't be that expensive anyway compared to the rest of what happens in the critical section.

        Show
        andrew.wang Andrew Wang added a comment - Thanks for reviewing Wei-chiu! The list of active policies is stored in a map, and the fsd lock protects it from concurrent access. I think we should probably keep it within the lock? A single treemap lookup shouldn't be that expensive anyway compared to the rest of what happens in the critical section.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Hi Andrew Wang thanks for posting such a big patch.
        Most of the change look good to me at my first pass of review, and I have just one comment:

        In FSDirErasureCodingOp#setErasureCodingPolicy
        You can get ecPolicy before acquiring FSDirectory write lock.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Hi Andrew Wang thanks for posting such a big patch. Most of the change look good to me at my first pass of review, and I have just one comment: In FSDirErasureCodingOp#setErasureCodingPolicy You can get ecPolicy before acquiring FSDirectory write lock.
        Hide
        andrew.wang Andrew Wang added a comment -

        Remaining test failures look like flakes, please review.

        Show
        andrew.wang Andrew Wang added a comment - Remaining test failures look like flakes, please review.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 22s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 37 new or modified test files.
        0 mvndep 2m 11s Maven dependency ordering for branch
        +1 mvninstall 13m 41s trunk passed
        +1 compile 12m 30s trunk passed
        +1 checkstyle 1m 55s trunk passed
        +1 mvnsite 2m 4s trunk passed
        +1 mvneclipse 0m 59s trunk passed
        +1 findbugs 3m 49s trunk passed
        +1 javadoc 1m 29s trunk passed
        0 mvndep 0m 14s Maven dependency ordering for patch
        +1 mvninstall 1m 39s the patch passed
        +1 compile 11m 52s the patch passed
        +1 cc 11m 52s the patch passed
        +1 javac 11m 52s the patch passed
        +1 checkstyle 1m 53s the patch passed
        +1 mvnsite 2m 15s the patch passed
        +1 mvneclipse 0m 58s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 2s The patch has no ill-formed XML file.
        +1 findbugs 4m 45s the patch passed
        +1 javadoc 1m 30s the patch passed
        +1 unit 1m 12s hadoop-hdfs-client in the patch passed.
        -1 unit 93m 7s hadoop-hdfs in the patch failed.
        +1 unit 113m 17s hadoop-mapreduce-client-jobclient in the patch passed.
        +1 asflicense 0m 45s The patch does not generate ASF License warnings.
        274m 29s



        Reason Tests
        Failed junit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure020
          hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11428
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12854405/HDFS-11428.003.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc xml
        uname Linux 42e6628c8177 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 132f758
        Default Java 1.8.0_121
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/18435/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18435/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: .
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18435/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 22s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 37 new or modified test files. 0 mvndep 2m 11s Maven dependency ordering for branch +1 mvninstall 13m 41s trunk passed +1 compile 12m 30s trunk passed +1 checkstyle 1m 55s trunk passed +1 mvnsite 2m 4s trunk passed +1 mvneclipse 0m 59s trunk passed +1 findbugs 3m 49s trunk passed +1 javadoc 1m 29s trunk passed 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 1m 39s the patch passed +1 compile 11m 52s the patch passed +1 cc 11m 52s the patch passed +1 javac 11m 52s the patch passed +1 checkstyle 1m 53s the patch passed +1 mvnsite 2m 15s the patch passed +1 mvneclipse 0m 58s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 findbugs 4m 45s the patch passed +1 javadoc 1m 30s the patch passed +1 unit 1m 12s hadoop-hdfs-client in the patch passed. -1 unit 93m 7s hadoop-hdfs in the patch failed. +1 unit 113m 17s hadoop-mapreduce-client-jobclient in the patch passed. +1 asflicense 0m 45s The patch does not generate ASF License warnings. 274m 29s Reason Tests Failed junit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure020   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11428 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12854405/HDFS-11428.003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc xml uname Linux 42e6628c8177 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 132f758 Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/18435/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18435/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18435/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        andrew.wang Andrew Wang added a comment -

        Missed some nulls in test files.

        Show
        andrew.wang Andrew Wang added a comment - Missed some nulls in test files.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 25s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 16 new or modified test files.
        0 mvndep 0m 27s Maven dependency ordering for branch
        +1 mvninstall 13m 47s trunk passed
        +1 compile 1m 30s trunk passed
        +1 checkstyle 0m 41s trunk passed
        +1 mvnsite 1m 37s trunk passed
        +1 mvneclipse 0m 25s trunk passed
        +1 findbugs 3m 35s trunk passed
        +1 javadoc 1m 12s trunk passed
        0 mvndep 0m 7s Maven dependency ordering for patch
        +1 mvninstall 1m 34s the patch passed
        +1 compile 1m 40s the patch passed
        +1 cc 1m 40s the patch passed
        +1 javac 1m 40s the patch passed
        +1 checkstyle 0m 46s the patch passed
        +1 mvnsite 1m 37s the patch passed
        +1 mvneclipse 0m 20s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 2s The patch has no ill-formed XML file.
        +1 findbugs 3m 54s the patch passed
        +1 javadoc 1m 6s the patch passed
        +1 unit 1m 5s hadoop-hdfs-client in the patch passed.
        -1 unit 90m 0s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 19s The patch does not generate ASF License warnings.
        128m 1s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped
          hadoop.hdfs.server.datanode.TestDataNodeErasureCodingMetrics
          hadoop.hdfs.TestDecommissionWithStriped
          hadoop.hdfs.server.balancer.TestBalancer
          hadoop.hdfs.TestReconstructStripedFile
          hadoop.hdfs.server.namenode.TestAddStripedBlocks
          hadoop.hdfs.server.namenode.TestAddStripedBlockInFBR
          hadoop.hdfs.server.namenode.TestReconstructStripedBlocks
          hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerWithStripedBlocks
          hadoop.hdfs.server.blockmanagement.TestReconstructStripedBlocksWithRackAwareness
          hadoop.hdfs.TestReadStripedFileWithDecoding
          hadoop.hdfs.server.blockmanagement.TestSequentialBlockGroupId
          hadoop.hdfs.server.namenode.TestNameNodeMXBean
          hadoop.hdfs.server.namenode.TestAddOverReplicatedStripedBlocks
          hadoop.hdfs.server.mover.TestMover
          hadoop.hdfs.TestErasureCodingPolicies
          hadoop.hdfs.TestSafeModeWithStripedFile
          hadoop.hdfs.server.namenode.TestStripedINodeFile
          hadoop.hdfs.TestWriteReadStripedFile
          hadoop.hdfs.server.namenode.TestFsck
          hadoop.hdfs.TestFileChecksum



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11428
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12854328/HDFS-11428.002.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc xml
        uname Linux 67faaeb5fe18 3.13.0-108-generic #155-Ubuntu SMP Wed Jan 11 16:58:52 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 132f758
        Default Java 1.8.0_121
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/18431/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18431/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18431/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 25s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 16 new or modified test files. 0 mvndep 0m 27s Maven dependency ordering for branch +1 mvninstall 13m 47s trunk passed +1 compile 1m 30s trunk passed +1 checkstyle 0m 41s trunk passed +1 mvnsite 1m 37s trunk passed +1 mvneclipse 0m 25s trunk passed +1 findbugs 3m 35s trunk passed +1 javadoc 1m 12s trunk passed 0 mvndep 0m 7s Maven dependency ordering for patch +1 mvninstall 1m 34s the patch passed +1 compile 1m 40s the patch passed +1 cc 1m 40s the patch passed +1 javac 1m 40s the patch passed +1 checkstyle 0m 46s the patch passed +1 mvnsite 1m 37s the patch passed +1 mvneclipse 0m 20s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 findbugs 3m 54s the patch passed +1 javadoc 1m 6s the patch passed +1 unit 1m 5s hadoop-hdfs-client in the patch passed. -1 unit 90m 0s hadoop-hdfs in the patch failed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 128m 1s Reason Tests Failed junit tests hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped   hadoop.hdfs.server.datanode.TestDataNodeErasureCodingMetrics   hadoop.hdfs.TestDecommissionWithStriped   hadoop.hdfs.server.balancer.TestBalancer   hadoop.hdfs.TestReconstructStripedFile   hadoop.hdfs.server.namenode.TestAddStripedBlocks   hadoop.hdfs.server.namenode.TestAddStripedBlockInFBR   hadoop.hdfs.server.namenode.TestReconstructStripedBlocks   hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerWithStripedBlocks   hadoop.hdfs.server.blockmanagement.TestReconstructStripedBlocksWithRackAwareness   hadoop.hdfs.TestReadStripedFileWithDecoding   hadoop.hdfs.server.blockmanagement.TestSequentialBlockGroupId   hadoop.hdfs.server.namenode.TestNameNodeMXBean   hadoop.hdfs.server.namenode.TestAddOverReplicatedStripedBlocks   hadoop.hdfs.server.mover.TestMover   hadoop.hdfs.TestErasureCodingPolicies   hadoop.hdfs.TestSafeModeWithStripedFile   hadoop.hdfs.server.namenode.TestStripedINodeFile   hadoop.hdfs.TestWriteReadStripedFile   hadoop.hdfs.server.namenode.TestFsck   hadoop.hdfs.TestFileChecksum Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11428 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12854328/HDFS-11428.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc xml uname Linux 67faaeb5fe18 3.13.0-108-generic #155-Ubuntu SMP Wed Jan 11 16:58:52 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 132f758 Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/18431/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18431/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18431/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        andrew.wang Andrew Wang added a comment -

        Patch attached, let's see what precommit thinks. Wei-Chiu Chuang / Kai Zheng would you mind reviewing?

        Show
        andrew.wang Andrew Wang added a comment - Patch attached, let's see what precommit thinks. Wei-Chiu Chuang / Kai Zheng would you mind reviewing?
        Hide
        andrew.wang Andrew Wang added a comment -

        WIP patch attached, this depends on HDFS-11426 and HDFS-11427 so not marking PA for now.

        Show
        andrew.wang Andrew Wang added a comment - WIP patch attached, this depends on HDFS-11426 and HDFS-11427 so not marking PA for now.

          People

          • Assignee:
            andrew.wang Andrew Wang
            Reporter:
            andrew.wang Andrew Wang
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development