Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: encryption
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      This JIRA is opened to track adding support of nested encryption zone based on Andrew Wang's comment for certain use cases.

      1. HDFS-9244.00.patch
        4 kB
        Zhe Zhang
      2. HDFS-9244.01.patch
        6 kB
        Zhe Zhang
      3. HDFS-9244.02.patch
        7 kB
        Zhe Zhang
      4. HDFS-9244.03.patch
        11 kB
        Zhe Zhang
      5. HDFS-9244.04.patch
        11 kB
        Zhe Zhang
      6. HDFS-9244.05.patch
        20 kB
        Zhe Zhang
      7. HDFS-9244.06.patch
        20 kB
        Zhe Zhang
      8. HDFS-9244.07.patch
        21 kB
        Zhe Zhang

        Issue Links

          Activity

          Hide
          zhz Zhe Zhang added a comment -

          Thanks Xiaoyu for reporting this! Attaching initial patch for the fix.

          Another task is to update documentations so that users have updated understanding and expectations on EZ. I'll create a separate JIRA for that.

          Show
          zhz Zhe Zhang added a comment - Thanks Xiaoyu for reporting this! Attaching initial patch for the fix. Another task is to update documentations so that users have updated understanding and expectations on EZ. I'll create a separate JIRA for that.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 8m 9s trunk passed
          +1 compile 0m 47s trunk passed with JDK v1.8.0_66
          +1 compile 0m 45s trunk passed with JDK v1.7.0_91
          +1 checkstyle 0m 17s trunk passed
          +1 mvnsite 0m 55s trunk passed
          +1 mvneclipse 0m 16s trunk passed
          +1 findbugs 1m 59s trunk passed
          +1 javadoc 1m 9s trunk passed with JDK v1.8.0_66
          +1 javadoc 1m 49s trunk passed with JDK v1.7.0_91
          +1 mvninstall 0m 48s the patch passed
          +1 compile 0m 37s the patch passed with JDK v1.8.0_66
          +1 javac 0m 37s the patch passed
          +1 compile 0m 41s the patch passed with JDK v1.7.0_91
          +1 javac 0m 41s the patch passed
          +1 checkstyle 0m 15s the patch passed
          +1 mvnsite 0m 50s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 2m 1s the patch passed
          +1 javadoc 1m 4s the patch passed with JDK v1.8.0_66
          +1 javadoc 1m 47s the patch passed with JDK v1.7.0_91
          -1 unit 67m 45s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
          -1 unit 67m 16s hadoop-hdfs in the patch failed with JDK v1.7.0_91.
          +1 asflicense 0m 28s Patch does not generate ASF License warnings.
          163m 2s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure210
            hadoop.cli.TestCryptoAdminCLI
          JDK v1.7.0_91 Failed junit tests hadoop.hdfs.TestFileAppend2
            hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes
            hadoop.hdfs.server.datanode.TestBlockReplacement
            hadoop.cli.TestCryptoAdminCLI
            hadoop.hdfs.TestErasureCodingPolicies
            hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12781035/HDFS-9244.00.patch
          JIRA Issue HDFS-9244
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 5caef151e5fe 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 52b7757
          Default Java 1.7.0_91
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14055/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14055/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14055/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14055/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14055/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Max memory used 75MB
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14055/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 9s trunk passed +1 compile 0m 47s trunk passed with JDK v1.8.0_66 +1 compile 0m 45s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 17s trunk passed +1 mvnsite 0m 55s trunk passed +1 mvneclipse 0m 16s trunk passed +1 findbugs 1m 59s trunk passed +1 javadoc 1m 9s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 49s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 48s the patch passed +1 compile 0m 37s the patch passed with JDK v1.8.0_66 +1 javac 0m 37s the patch passed +1 compile 0m 41s the patch passed with JDK v1.7.0_91 +1 javac 0m 41s the patch passed +1 checkstyle 0m 15s the patch passed +1 mvnsite 0m 50s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 1s the patch passed +1 javadoc 1m 4s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 47s the patch passed with JDK v1.7.0_91 -1 unit 67m 45s hadoop-hdfs in the patch failed with JDK v1.8.0_66. -1 unit 67m 16s hadoop-hdfs in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 28s Patch does not generate ASF License warnings. 163m 2s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure210   hadoop.cli.TestCryptoAdminCLI JDK v1.7.0_91 Failed junit tests hadoop.hdfs.TestFileAppend2   hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes   hadoop.hdfs.server.datanode.TestBlockReplacement   hadoop.cli.TestCryptoAdminCLI   hadoop.hdfs.TestErasureCodingPolicies   hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12781035/HDFS-9244.00.patch JIRA Issue HDFS-9244 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 5caef151e5fe 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 52b7757 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/14055/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14055/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14055/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14055/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14055/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 75MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14055/console This message was automatically generated.
          Hide
          zhz Zhe Zhang added a comment -

          Updating the patch to fix test failure in TestCryptoAdminCLI. It was assuming the old EZ behavior (cannot create nested EZ).

          Show
          zhz Zhe Zhang added a comment - Updating the patch to fix test failure in TestCryptoAdminCLI . It was assuming the old EZ behavior (cannot create nested EZ).
          Hide
          xyao Xiaoyu Yao added a comment -

          Thanks Zhe Zhang for working on this. Can we clarify the use cases (in addition to the original one mentioned in the description) before unblocking this? And how often are they being used/requested by the customer deployments.

          My concern is that this could bring up tricky cases such as upgrade/rollback, trash, etc. to document, support and maintain for nested zones. We don't want to introduce unnecessary complexity unless there are important use cases behind it. Thanks!

          Show
          xyao Xiaoyu Yao added a comment - Thanks Zhe Zhang for working on this. Can we clarify the use cases (in addition to the original one mentioned in the description) before unblocking this? And how often are they being used/requested by the customer deployments. My concern is that this could bring up tricky cases such as upgrade/rollback, trash, etc. to document, support and maintain for nested zones. We don't want to introduce unnecessary complexity unless there are important use cases behind it. Thanks!
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 8m 11s trunk passed
          +1 compile 0m 40s trunk passed with JDK v1.8.0_66
          +1 compile 0m 43s trunk passed with JDK v1.7.0_91
          +1 checkstyle 0m 16s trunk passed
          +1 mvnsite 0m 52s trunk passed
          +1 mvneclipse 0m 15s trunk passed
          +1 findbugs 1m 55s trunk passed
          +1 javadoc 1m 4s trunk passed with JDK v1.8.0_66
          +1 javadoc 1m 47s trunk passed with JDK v1.7.0_91
          +1 mvninstall 0m 46s the patch passed
          +1 compile 0m 35s the patch passed with JDK v1.8.0_66
          +1 javac 0m 35s the patch passed
          +1 compile 0m 40s the patch passed with JDK v1.7.0_91
          +1 javac 0m 40s the patch passed
          +1 checkstyle 0m 16s the patch passed
          +1 mvnsite 0m 50s the patch passed
          +1 mvneclipse 0m 11s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 2m 2s the patch passed
          +1 javadoc 1m 6s the patch passed with JDK v1.8.0_66
          +1 javadoc 1m 47s the patch passed with JDK v1.7.0_91
          -1 unit 66m 11s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
          -1 unit 67m 31s hadoop-hdfs in the patch failed with JDK v1.7.0_91.
          +1 asflicense 0m 23s Patch does not generate ASF License warnings.
          160m 50s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.hdfs.TestDFSUpgradeFromImage
            hadoop.hdfs.server.datanode.TestFsDatasetCache
          JDK v1.7.0_91 Failed junit tests hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12781676/HDFS-9244.01.patch
          JIRA Issue HDFS-9244
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 2c083c445647 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / de37f37
          Default Java 1.7.0_91
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14093/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14093/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14093/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14093/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14093/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Max memory used 75MB
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14093/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 8m 11s trunk passed +1 compile 0m 40s trunk passed with JDK v1.8.0_66 +1 compile 0m 43s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 16s trunk passed +1 mvnsite 0m 52s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 1m 55s trunk passed +1 javadoc 1m 4s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 47s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 46s the patch passed +1 compile 0m 35s the patch passed with JDK v1.8.0_66 +1 javac 0m 35s the patch passed +1 compile 0m 40s the patch passed with JDK v1.7.0_91 +1 javac 0m 40s the patch passed +1 checkstyle 0m 16s the patch passed +1 mvnsite 0m 50s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 2m 2s the patch passed +1 javadoc 1m 6s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 47s the patch passed with JDK v1.7.0_91 -1 unit 66m 11s hadoop-hdfs in the patch failed with JDK v1.8.0_66. -1 unit 67m 31s hadoop-hdfs in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 23s Patch does not generate ASF License warnings. 160m 50s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.hdfs.TestDFSUpgradeFromImage   hadoop.hdfs.server.datanode.TestFsDatasetCache JDK v1.7.0_91 Failed junit tests hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12781676/HDFS-9244.01.patch JIRA Issue HDFS-9244 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 2c083c445647 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / de37f37 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/14093/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14093/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14093/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14093/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14093/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 75MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14093/console This message was automatically generated.
          Hide
          zhz Zhe Zhang added a comment -

          Xiaoyu Yao Thanks for the comment. The most important use case we've seen is "always-on encryption" – the ability to setup / as an EZ plus the flexibility to use different keys for descendant dirs later on. It has been requested by many customers. The Trash use case is also useful – it's not directly supported by this change, but will require the concept of nested EZ.

          Thanks for pointing out the possible issues. Upgrade won't cause issues, because a lower-versioned cluster with non-nested EZ setup is always legitimate under the new logic. Rollback won't be allowed from a nested-EZ cluster to version < 2.8. The actual logic change is very simple – basically just the removal of an if check. So I don't think the change itself is hard to support and maintain. The most challenging task is to clearly document the new semantics so that external tools don't rely on the assumption that all data under an EZ have the same key. I created HDFS-9644 to track the effort.

          Show
          zhz Zhe Zhang added a comment - Xiaoyu Yao Thanks for the comment. The most important use case we've seen is "always-on encryption" – the ability to setup / as an EZ plus the flexibility to use different keys for descendant dirs later on. It has been requested by many customers. The Trash use case is also useful – it's not directly supported by this change, but will require the concept of nested EZ. Thanks for pointing out the possible issues. Upgrade won't cause issues, because a lower-versioned cluster with non-nested EZ setup is always legitimate under the new logic. Rollback won't be allowed from a nested-EZ cluster to version < 2.8. The actual logic change is very simple – basically just the removal of an if check. So I don't think the change itself is hard to support and maintain. The most challenging task is to clearly document the new semantics so that external tools don't rely on the assumption that all data under an EZ have the same key. I created HDFS-9644 to track the effort.
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          > ... Rollback won't be allowed ...

          Rollback must always be allowed for any feature. It is for protecting user data against upgrade failure due to, mostly likely, user errors and, less likely, software bug in the new version.

          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - > ... Rollback won't be allowed ... Rollback must always be allowed for any feature. It is for protecting user data against upgrade failure due to, mostly likely, user errors and, less likely, software bug in the new version.
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          Hi Zhe Zhang, will this fix break the EZ trash support introduced by HDFS-8831?

          Show
          arpitagarwal Arpit Agarwal added a comment - Hi Zhe Zhang , will this fix break the EZ trash support introduced by HDFS-8831 ?
          Hide
          sanjay.radia Sanjay Radia added a comment -

          The main motivation for nested EZ is root + subdirs as per Andrew's comment. Is it such a big deal for an admin to set up EZ as he creates the directories in dirs? I think nested encryption will complicate things like volumes down the road and I don't think this extra complexity is necessary.
          I will comment the volumes jira drive that discussion to a conclusion.

          Show
          sanjay.radia Sanjay Radia added a comment - The main motivation for nested EZ is root + subdirs as per Andrew's comment. Is it such a big deal for an admin to set up EZ as he creates the directories in dirs? I think nested encryption will complicate things like volumes down the road and I don't think this extra complexity is necessary. I will comment the volumes jira drive that discussion to a conclusion.
          Hide
          zhz Zhe Zhang added a comment -

          Thanks for the helpful comments Tsz Wo Nicholas Sze, Arpit Agarwal, and Sanjay Radia! Really appreciate the discussion.

          Rollback must always be allowed for any feature.

          Sorry I used "rollback" when I meant "downgrade". With the change, downgrade won't be allowed; I will update the patch to bump the NN layout version. Rollback will work fine.

          will this fix break the EZ trash support introduced by HDFS-8831?

          No it won't break Trash support:

          The deleted encrypted files will remain encrypted and be moved to .Trash subdirectory under the root of the encryption zone

          With nested EZs, the "root of the EZ" will be the nearest ancestor with an EZ setting. I will extend the HDFS-8831 unit test to demonstrate this.

          The main motivation for nested EZ is root + subdirs as per Andrew's comment.

          Yes this is the main motivation.

          Is it such a big deal for an admin to set up EZ as he creates the directories in dirs?

          We have received many requests from admins for this support. I think this is a natural trend as the encryption feature matures and people use it in more sophisticated ways.

          I think nested encryption will complicate things like volumes

          I look forward to more discussions under HDFS-8888. Meanwhile, as Andrew commented , the complexity of directory-level EZ has already been implemented. The additional complexity introduced by this change is really minimum: it basically only relaxes one if condition check.

          From the ease-of-administration perspective, I think even with nested EZs (and other nested policies like erasure coding), we can always enforce volume-level uniformity. E.g. we can just disallow creating EZ (or setting EC policy) on a file/dir in a volume.

          Show
          zhz Zhe Zhang added a comment - Thanks for the helpful comments Tsz Wo Nicholas Sze , Arpit Agarwal , and Sanjay Radia ! Really appreciate the discussion. Rollback must always be allowed for any feature. Sorry I used "rollback" when I meant "downgrade". With the change, downgrade won't be allowed; I will update the patch to bump the NN layout version. Rollback will work fine. will this fix break the EZ trash support introduced by HDFS-8831 ? No it won't break Trash support: The deleted encrypted files will remain encrypted and be moved to .Trash subdirectory under the root of the encryption zone With nested EZs, the "root of the EZ" will be the nearest ancestor with an EZ setting. I will extend the HDFS-8831 unit test to demonstrate this. The main motivation for nested EZ is root + subdirs as per Andrew's comment. Yes this is the main motivation. Is it such a big deal for an admin to set up EZ as he creates the directories in dirs? We have received many requests from admins for this support. I think this is a natural trend as the encryption feature matures and people use it in more sophisticated ways. I think nested encryption will complicate things like volumes I look forward to more discussions under HDFS-8888 . Meanwhile, as Andrew commented , the complexity of directory-level EZ has already been implemented. The additional complexity introduced by this change is really minimum: it basically only relaxes one if condition check. From the ease-of-administration perspective, I think even with nested EZs (and other nested policies like erasure coding), we can always enforce volume-level uniformity. E.g. we can just disallow creating EZ (or setting EC policy) on a file/dir in a volume.
          Hide
          zhz Zhe Zhang added a comment -

          Extending testEncryptionZoneWithTrash to verify behavior of nested EZs.

          Assume we have /topEZ/nestedEZ. The current Trash logic (in particular DistributedFileSystem#getTrashRoot), considers nestedEZ to be root dir of an EZ (ez.getPath().equals(absSrc)). Therefore when we remove nesteEZ it goes to /users/username/.Trash/topEZ/nestedEZ. As part of the rm, {{/users/username/.Trash/topEZ will be created. Later when we rm /topEZ, there will be a name clash and /users/username/.Trash/topEZxxxx will be created, xxxx is timestamp. So all deleted data is still in Trash, but additional recovery logic is needed.

          If we want to avoid this we should update DistributedFileSystem#getTrashRoot to check if the parent of the given path is in an EZ (or is an EZ itself). We can either add the logic in getTrashRoot or add an API in EncryptionZoneManager. Once we reach an agreement I'll update the patch.

          Show
          zhz Zhe Zhang added a comment - Extending testEncryptionZoneWithTrash to verify behavior of nested EZs. Assume we have /topEZ/nestedEZ . The current Trash logic (in particular DistributedFileSystem#getTrashRoot ), considers nestedEZ to be root dir of an EZ ( ez.getPath().equals(absSrc) ). Therefore when we remove nesteEZ it goes to /users/username/.Trash/topEZ/nestedEZ . As part of the rm, {{ /users/username/.Trash/topEZ will be created. Later when we rm /topEZ , there will be a name clash and /users/username/.Trash/topEZxxxx will be created, xxxx is timestamp. So all deleted data is still in Trash, but additional recovery logic is needed. If we want to avoid this we should update DistributedFileSystem#getTrashRoot to check if the parent of the given path is in an EZ (or is an EZ itself). We can either add the logic in getTrashRoot or add an API in EncryptionZoneManager . Once we reach an agreement I'll update the patch.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 11m 14s trunk passed
          +1 compile 1m 33s trunk passed with JDK v1.8.0_66
          +1 compile 1m 9s trunk passed with JDK v1.7.0_91
          +1 checkstyle 0m 29s trunk passed
          +1 mvnsite 1m 25s trunk passed
          +1 mvneclipse 0m 18s trunk passed
          +1 findbugs 2m 57s trunk passed
          +1 javadoc 2m 0s trunk passed with JDK v1.8.0_66
          +1 javadoc 3m 1s trunk passed with JDK v1.7.0_91
          +1 mvninstall 1m 15s the patch passed
          +1 compile 1m 29s the patch passed with JDK v1.8.0_66
          +1 javac 1m 29s the patch passed
          +1 compile 1m 10s the patch passed with JDK v1.7.0_91
          +1 javac 1m 10s the patch passed
          +1 checkstyle 0m 25s the patch passed
          +1 mvnsite 1m 25s the patch passed
          +1 mvneclipse 0m 17s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 0s The patch has no ill-formed XML file.
          +1 findbugs 3m 8s the patch passed
          +1 javadoc 2m 2s the patch passed with JDK v1.8.0_66
          +1 javadoc 2m 54s the patch passed with JDK v1.7.0_91
          -1 unit 80m 19s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
          -1 unit 79m 49s hadoop-hdfs in the patch failed with JDK v1.7.0_91.
          -1 asflicense 0m 24s Patch generated 2 ASF License warnings.
          202m 35s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.hdfs.server.datanode.TestDirectoryScanner
            hadoop.hdfs.TestEncryptionZones
            hadoop.hdfs.qjournal.TestSecureNNWithQJM
            hadoop.hdfs.server.namenode.ha.TestEditLogTailer
            hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
            hadoop.hdfs.security.TestDelegationTokenForProxyUser
            hadoop.hdfs.TestFileAppend
            hadoop.hdfs.TestDFSUpgradeFromImage
            hadoop.hdfs.TestEncryptionZonesWithKMS
            hadoop.hdfs.server.namenode.ha.TestRequestHedgingProxyProvider
            hadoop.hdfs.TestDataTransferKeepalive
          JDK v1.8.0_66 Timed out junit tests org.apache.hadoop.hdfs.TestFileCreation
            org.apache.hadoop.hdfs.TestDFSClientRetries
            org.apache.hadoop.hdfs.server.balancer.TestBalancer
            org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner
          JDK v1.7.0_91 Failed junit tests hadoop.hdfs.TestEncryptionZones
            hadoop.hdfs.server.namenode.TestDecommissioningStatus
            hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery
            hadoop.hdfs.TestEncryptionZonesWithKMS



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12785304/HDFS-9244.02.patch
          JIRA Issue HDFS-9244
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 501df85c4507 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 772ea7b
          Default Java 1.7.0_91
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14296/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14296/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14296/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14296/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14296/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/14296/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Max memory used 76MB
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14296/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 11m 14s trunk passed +1 compile 1m 33s trunk passed with JDK v1.8.0_66 +1 compile 1m 9s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 29s trunk passed +1 mvnsite 1m 25s trunk passed +1 mvneclipse 0m 18s trunk passed +1 findbugs 2m 57s trunk passed +1 javadoc 2m 0s trunk passed with JDK v1.8.0_66 +1 javadoc 3m 1s trunk passed with JDK v1.7.0_91 +1 mvninstall 1m 15s the patch passed +1 compile 1m 29s the patch passed with JDK v1.8.0_66 +1 javac 1m 29s the patch passed +1 compile 1m 10s the patch passed with JDK v1.7.0_91 +1 javac 1m 10s the patch passed +1 checkstyle 0m 25s the patch passed +1 mvnsite 1m 25s the patch passed +1 mvneclipse 0m 17s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 0s The patch has no ill-formed XML file. +1 findbugs 3m 8s the patch passed +1 javadoc 2m 2s the patch passed with JDK v1.8.0_66 +1 javadoc 2m 54s the patch passed with JDK v1.7.0_91 -1 unit 80m 19s hadoop-hdfs in the patch failed with JDK v1.8.0_66. -1 unit 79m 49s hadoop-hdfs in the patch failed with JDK v1.7.0_91. -1 asflicense 0m 24s Patch generated 2 ASF License warnings. 202m 35s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.hdfs.server.datanode.TestDirectoryScanner   hadoop.hdfs.TestEncryptionZones   hadoop.hdfs.qjournal.TestSecureNNWithQJM   hadoop.hdfs.server.namenode.ha.TestEditLogTailer   hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints   hadoop.hdfs.security.TestDelegationTokenForProxyUser   hadoop.hdfs.TestFileAppend   hadoop.hdfs.TestDFSUpgradeFromImage   hadoop.hdfs.TestEncryptionZonesWithKMS   hadoop.hdfs.server.namenode.ha.TestRequestHedgingProxyProvider   hadoop.hdfs.TestDataTransferKeepalive JDK v1.8.0_66 Timed out junit tests org.apache.hadoop.hdfs.TestFileCreation   org.apache.hadoop.hdfs.TestDFSClientRetries   org.apache.hadoop.hdfs.server.balancer.TestBalancer   org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner JDK v1.7.0_91 Failed junit tests hadoop.hdfs.TestEncryptionZones   hadoop.hdfs.server.namenode.TestDecommissioningStatus   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery   hadoop.hdfs.TestEncryptionZonesWithKMS Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12785304/HDFS-9244.02.patch JIRA Issue HDFS-9244 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 501df85c4507 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 772ea7b Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/14296/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14296/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14296/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14296/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14296/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/14296/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14296/console This message was automatically generated.
          Hide
          zhz Zhe Zhang added a comment -

          Updating the patch per comments from Xiaoyu and Andrew under HDFS-9688.

          I think for both movability check in general, and Trash support in specific, what really matters is the parent EZ of a path. Instead of getting the EZ of a path and checking if the EZ is equal to the path itself, we should just directly get the parent EZ.

          One caveat is whether we should allow moving the root dir of a nested EZ itself. E.g., should mv /topEZ/nestedEZ /nesteEZ be allowed? How about we disallow it in this initial patch, and revisit if such flexibility is asked for?

          Show
          zhz Zhe Zhang added a comment - Updating the patch per comments from Xiaoyu and Andrew under HDFS-9688 . I think for both movability check in general, and Trash support in specific, what really matters is the parent EZ of a path. Instead of getting the EZ of a path and checking if the EZ is equal to the path itself, we should just directly get the parent EZ. One caveat is whether we should allow moving the root dir of a nested EZ itself. E.g., should mv /topEZ/nestedEZ /nesteEZ be allowed? How about we disallow it in this initial patch, and revisit if such flexibility is asked for?
          Hide
          andrew.wang Andrew Wang added a comment -

          Thanks for working on this Zhe, some code review comments:

          • getParentEZForPath, we could call getParentINodesInPath and pass it to getEZForPath right? That way we don't have a copy paste. One other note is that "/" is its own parent but right now we return null. Since you can't rename "/" or create a parent for "/" I don't know if this matters.
          • In createEncryptionZone a nice improvement might be checking that the src exists and isDir and throwing FileNotFoundException if not. The logic is weirdly nested right now, what I'd expect is a sequence of {{if (condition} { throw; }

            }}. I think this is because we wait until the unprotectedSetXAttrs at the end to throw FNF.

          • IMO rewording the error message as "Directory %s is already an encryption zone." is more clear.
          • I remember there being a new TestNestedEZ file, which I assume will make a reappearance before we commit this. That'd be a good place to do the trash-related tests too.

          I'll play with the unit tests a bit myself to get a feel for what is and isn't allowed, implementation wise it looks like what we discussed on that other JIRA.

          I'm also okay with the caveat you stated, at first blush it doesn't affect trash support, which is our big concern throughout this.

          Show
          andrew.wang Andrew Wang added a comment - Thanks for working on this Zhe, some code review comments: getParentEZForPath, we could call getParentINodesInPath and pass it to getEZForPath right? That way we don't have a copy paste. One other note is that "/" is its own parent but right now we return null. Since you can't rename "/" or create a parent for "/" I don't know if this matters. In createEncryptionZone a nice improvement might be checking that the src exists and isDir and throwing FileNotFoundException if not. The logic is weirdly nested right now, what I'd expect is a sequence of {{if (condition} { throw; } }}. I think this is because we wait until the unprotectedSetXAttrs at the end to throw FNF. IMO rewording the error message as "Directory %s is already an encryption zone." is more clear. I remember there being a new TestNestedEZ file, which I assume will make a reappearance before we commit this. That'd be a good place to do the trash-related tests too. I'll play with the unit tests a bit myself to get a feel for what is and isn't allowed, implementation wise it looks like what we discussed on that other JIRA. I'm also okay with the caveat you stated, at first blush it doesn't affect trash support, which is our big concern throughout this.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          0 mvndep 0m 25s Maven dependency ordering for branch
          +1 mvninstall 7m 36s trunk passed
          +1 compile 1m 31s trunk passed with JDK v1.8.0_66
          +1 compile 1m 35s trunk passed with JDK v1.7.0_91
          +1 checkstyle 0m 30s trunk passed
          +1 mvnsite 1m 35s trunk passed
          +1 mvneclipse 0m 26s trunk passed
          +1 findbugs 4m 4s trunk passed
          +1 javadoc 1m 37s trunk passed with JDK v1.8.0_66
          +1 javadoc 2m 24s trunk passed with JDK v1.7.0_91
          0 mvndep 0m 17s Maven dependency ordering for patch
          +1 mvninstall 1m 26s the patch passed
          +1 compile 1m 31s the patch passed with JDK v1.8.0_66
          +1 javac 1m 31s the patch passed
          +1 compile 1m 28s the patch passed with JDK v1.7.0_91
          +1 javac 1m 28s the patch passed
          +1 checkstyle 0m 25s the patch passed
          +1 mvnsite 1m 32s the patch passed
          +1 mvneclipse 0m 25s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          -1 findbugs 2m 25s hadoop-hdfs-project/hadoop-hdfs-client generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
          +1 javadoc 1m 29s the patch passed with JDK v1.8.0_66
          +1 javadoc 2m 31s the patch passed with JDK v1.7.0_91
          +1 unit 0m 49s hadoop-hdfs-client in the patch passed with JDK v1.8.0_66.
          -1 unit 59m 1s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
          +1 unit 1m 0s hadoop-hdfs-client in the patch passed with JDK v1.7.0_91.
          +1 unit 49m 31s hadoop-hdfs in the patch passed with JDK v1.7.0_91.
          +1 asflicense 0m 22s Patch does not generate ASF License warnings.
          150m 58s



          Reason Tests
          FindBugs module:hadoop-hdfs-project/hadoop-hdfs-client
            Dead store to absSrc in org.apache.hadoop.hdfs.DistributedFileSystem.getTrashRoot(Path) At DistributedFileSystem.java:org.apache.hadoop.hdfs.DistributedFileSystem.getTrashRoot(Path) At DistributedFileSystem.java:[line 2353]
          JDK v1.8.0_66 Failed junit tests hadoop.hdfs.TestCrcCorruption
            hadoop.hdfs.server.datanode.TestBlockScanner



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12785656/HDFS-9244.03.patch
          JIRA Issue HDFS-9244
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 6a977b19036b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / ed55950
          Default Java 1.7.0_91
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
          findbugs v3.0.0
          findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/14331/artifact/patchprocess/new-findbugs-hadoop-hdfs-project_hadoop-hdfs-client.html
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14331/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14331/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14331/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
          Max memory used 76MB
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14331/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 25s Maven dependency ordering for branch +1 mvninstall 7m 36s trunk passed +1 compile 1m 31s trunk passed with JDK v1.8.0_66 +1 compile 1m 35s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 30s trunk passed +1 mvnsite 1m 35s trunk passed +1 mvneclipse 0m 26s trunk passed +1 findbugs 4m 4s trunk passed +1 javadoc 1m 37s trunk passed with JDK v1.8.0_66 +1 javadoc 2m 24s trunk passed with JDK v1.7.0_91 0 mvndep 0m 17s Maven dependency ordering for patch +1 mvninstall 1m 26s the patch passed +1 compile 1m 31s the patch passed with JDK v1.8.0_66 +1 javac 1m 31s the patch passed +1 compile 1m 28s the patch passed with JDK v1.7.0_91 +1 javac 1m 28s the patch passed +1 checkstyle 0m 25s the patch passed +1 mvnsite 1m 32s the patch passed +1 mvneclipse 0m 25s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. -1 findbugs 2m 25s hadoop-hdfs-project/hadoop-hdfs-client generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) +1 javadoc 1m 29s the patch passed with JDK v1.8.0_66 +1 javadoc 2m 31s the patch passed with JDK v1.7.0_91 +1 unit 0m 49s hadoop-hdfs-client in the patch passed with JDK v1.8.0_66. -1 unit 59m 1s hadoop-hdfs in the patch failed with JDK v1.8.0_66. +1 unit 1m 0s hadoop-hdfs-client in the patch passed with JDK v1.7.0_91. +1 unit 49m 31s hadoop-hdfs in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 22s Patch does not generate ASF License warnings. 150m 58s Reason Tests FindBugs module:hadoop-hdfs-project/hadoop-hdfs-client   Dead store to absSrc in org.apache.hadoop.hdfs.DistributedFileSystem.getTrashRoot(Path) At DistributedFileSystem.java:org.apache.hadoop.hdfs.DistributedFileSystem.getTrashRoot(Path) At DistributedFileSystem.java: [line 2353] JDK v1.8.0_66 Failed junit tests hadoop.hdfs.TestCrcCorruption   hadoop.hdfs.server.datanode.TestBlockScanner Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12785656/HDFS-9244.03.patch JIRA Issue HDFS-9244 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 6a977b19036b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ed55950 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/14331/artifact/patchprocess/new-findbugs-hadoop-hdfs-project_hadoop-hdfs-client.html unit https://builds.apache.org/job/PreCommit-HDFS-Build/14331/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14331/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14331/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14331/console This message was automatically generated.
          Hide
          zhz Zhe Zhang added a comment -

          Thanks Andrew Wang! Very helpful reviews. Attaching new patch to address the comments. The new TestNestedEZ is in the HDFS-9688 patch. It's a little tricky to include it here, because this change is targeted for 2.8, while the test targets for 2.6/2.7. Trash behavior under nested EZ is tested in testEncryptionZoneWithTrash.

          Show
          zhz Zhe Zhang added a comment - Thanks Andrew Wang ! Very helpful reviews. Attaching new patch to address the comments. The new TestNestedEZ is in the HDFS-9688 patch. It's a little tricky to include it here, because this change is targeted for 2.8, while the test targets for 2.6/2.7. Trash behavior under nested EZ is tested in testEncryptionZoneWithTrash .
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          0 mvndep 0m 23s Maven dependency ordering for branch
          +1 mvninstall 10m 36s trunk passed
          +1 compile 2m 57s trunk passed with JDK v1.8.0_66
          +1 compile 2m 15s trunk passed with JDK v1.7.0_91
          +1 checkstyle 0m 38s trunk passed
          +1 mvnsite 2m 19s trunk passed
          +1 mvneclipse 0m 37s trunk passed
          +1 findbugs 5m 41s trunk passed
          +1 javadoc 2m 38s trunk passed with JDK v1.8.0_66
          +1 javadoc 3m 35s trunk passed with JDK v1.7.0_91
          0 mvndep 0m 23s Maven dependency ordering for patch
          +1 mvninstall 2m 7s the patch passed
          +1 compile 2m 56s the patch passed with JDK v1.8.0_66
          +1 javac 2m 56s the patch passed
          +1 compile 2m 15s the patch passed with JDK v1.7.0_91
          +1 javac 2m 15s the patch passed
          +1 checkstyle 0m 35s the patch passed
          +1 mvnsite 2m 10s the patch passed
          +1 mvneclipse 0m 32s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 6m 21s the patch passed
          +1 javadoc 2m 28s the patch passed with JDK v1.8.0_66
          +1 javadoc 3m 33s the patch passed with JDK v1.7.0_91
          +1 unit 1m 33s hadoop-hdfs-client in the patch passed with JDK v1.8.0_66.
          -1 unit 100m 12s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
          +1 unit 1m 26s hadoop-hdfs-client in the patch passed with JDK v1.7.0_91.
          -1 unit 87m 48s hadoop-hdfs in the patch failed with JDK v1.7.0_91.
          +1 asflicense 0m 31s Patch does not generate ASF License warnings.
          250m 44s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.hdfs.qjournal.TestSecureNNWithQJM
            hadoop.hdfs.TestRecoverStripedFile
            hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl
            hadoop.hdfs.TestFileAppend
            hadoop.hdfs.server.namenode.TestFileTruncate
            hadoop.hdfs.server.datanode.TestBlockScanner
            hadoop.hdfs.server.mover.TestStorageMover
            hadoop.hdfs.server.namenode.ha.TestEditLogTailer
            hadoop.hdfs.server.datanode.TestDirectoryScanner
            hadoop.hdfs.server.namenode.ha.TestHAAppend
            hadoop.hdfs.server.namenode.TestRecoverStripedBlocks
            hadoop.hdfs.security.TestDelegationTokenForProxyUser
          JDK v1.7.0_91 Failed junit tests hadoop.hdfs.TestSafeModeWithStripedFile
            hadoop.hdfs.qjournal.TestSecureNNWithQJM
            hadoop.fs.viewfs.TestViewFsHdfs
            hadoop.hdfs.server.datanode.TestDirectoryScanner
            hadoop.fs.TestSymlinkHdfsFileContext
            hadoop.hdfs.security.TestDelegationTokenForProxyUser



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12785734/HDFS-9244.04.patch
          JIRA Issue HDFS-9244
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 8484a2c5b148 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 4d7055e
          Default Java 1.7.0_91
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14337/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14337/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14337/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14337/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14337/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
          Max memory used 76MB
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14337/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 23s Maven dependency ordering for branch +1 mvninstall 10m 36s trunk passed +1 compile 2m 57s trunk passed with JDK v1.8.0_66 +1 compile 2m 15s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 38s trunk passed +1 mvnsite 2m 19s trunk passed +1 mvneclipse 0m 37s trunk passed +1 findbugs 5m 41s trunk passed +1 javadoc 2m 38s trunk passed with JDK v1.8.0_66 +1 javadoc 3m 35s trunk passed with JDK v1.7.0_91 0 mvndep 0m 23s Maven dependency ordering for patch +1 mvninstall 2m 7s the patch passed +1 compile 2m 56s the patch passed with JDK v1.8.0_66 +1 javac 2m 56s the patch passed +1 compile 2m 15s the patch passed with JDK v1.7.0_91 +1 javac 2m 15s the patch passed +1 checkstyle 0m 35s the patch passed +1 mvnsite 2m 10s the patch passed +1 mvneclipse 0m 32s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 6m 21s the patch passed +1 javadoc 2m 28s the patch passed with JDK v1.8.0_66 +1 javadoc 3m 33s the patch passed with JDK v1.7.0_91 +1 unit 1m 33s hadoop-hdfs-client in the patch passed with JDK v1.8.0_66. -1 unit 100m 12s hadoop-hdfs in the patch failed with JDK v1.8.0_66. +1 unit 1m 26s hadoop-hdfs-client in the patch passed with JDK v1.7.0_91. -1 unit 87m 48s hadoop-hdfs in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 31s Patch does not generate ASF License warnings. 250m 44s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.hdfs.qjournal.TestSecureNNWithQJM   hadoop.hdfs.TestRecoverStripedFile   hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl   hadoop.hdfs.TestFileAppend   hadoop.hdfs.server.namenode.TestFileTruncate   hadoop.hdfs.server.datanode.TestBlockScanner   hadoop.hdfs.server.mover.TestStorageMover   hadoop.hdfs.server.namenode.ha.TestEditLogTailer   hadoop.hdfs.server.datanode.TestDirectoryScanner   hadoop.hdfs.server.namenode.ha.TestHAAppend   hadoop.hdfs.server.namenode.TestRecoverStripedBlocks   hadoop.hdfs.security.TestDelegationTokenForProxyUser JDK v1.7.0_91 Failed junit tests hadoop.hdfs.TestSafeModeWithStripedFile   hadoop.hdfs.qjournal.TestSecureNNWithQJM   hadoop.fs.viewfs.TestViewFsHdfs   hadoop.hdfs.server.datanode.TestDirectoryScanner   hadoop.fs.TestSymlinkHdfsFileContext   hadoop.hdfs.security.TestDelegationTokenForProxyUser Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12785734/HDFS-9244.04.patch JIRA Issue HDFS-9244 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 8484a2c5b148 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 4d7055e Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/14337/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14337/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14337/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14337/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14337/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14337/console This message was automatically generated.
          Hide
          zhz Zhe Zhang added a comment -

          The patch only takes effect when EZ or Trash is used. None of the above reported failures are related.

          Show
          zhz Zhe Zhang added a comment - The patch only takes effect when EZ or Trash is used. None of the above reported failures are related.
          Hide
          andrew.wang Andrew Wang added a comment -

          Thanks for the rev, I think we're close:

          • Can we add the same preconditions from getEncryptionZoneForPath to the the getParentEZ method? meaning the javadoc about the needing to hold the lock, the lock assert, the precondition check. The lock javadoc also should refer to the FSDirectory lock, there's no manager lock (please fix the existing javadoc on getEZForPath too while you're at it).
          • In createEZ, let's throw FileNotFoundException rather than IOException in the new check, better to have our exceptions be specific if possible.
          • I do still expect more extensive test coverage of nested encryption zones beyond trash deletes. Some non-exhaustive examples: ensuring correct key is used when nested, renames within a nested EZ work, renaming a nested file to the parent EZ does not work, renaming above the parent EZ does not work, renaming a nested root within the parent and outside the parent works, renaming a non-nested EZ inside an EZ works.
          • Update checkMoveValidity javadoc with these intended behaviors.
          Show
          andrew.wang Andrew Wang added a comment - Thanks for the rev, I think we're close: Can we add the same preconditions from getEncryptionZoneForPath to the the getParentEZ method? meaning the javadoc about the needing to hold the lock, the lock assert, the precondition check. The lock javadoc also should refer to the FSDirectory lock, there's no manager lock (please fix the existing javadoc on getEZForPath too while you're at it). In createEZ, let's throw FileNotFoundException rather than IOException in the new check, better to have our exceptions be specific if possible. I do still expect more extensive test coverage of nested encryption zones beyond trash deletes. Some non-exhaustive examples: ensuring correct key is used when nested, renames within a nested EZ work, renaming a nested file to the parent EZ does not work, renaming above the parent EZ does not work, renaming a nested root within the parent and outside the parent works, renaming a non-nested EZ inside an EZ works. Update checkMoveValidity javadoc with these intended behaviors.
          Hide
          zhz Zhe Zhang added a comment -

          Thanks Andrew for the review, it's very helpful. Updating the patch to address the comments.

          renaming a nested root within the parent and outside the parent works, renaming a non-nested EZ inside an EZ works.

          Renaming the root dir of a nested EZ outside the parent EZ is not supported now (the "caveat" we discussed above).

          Can you elaborate a bit on "renaming a non-nested EZ inside an EZ"?

          Show
          zhz Zhe Zhang added a comment - Thanks Andrew for the review, it's very helpful. Updating the patch to address the comments. renaming a nested root within the parent and outside the parent works, renaming a non-nested EZ inside an EZ works. Renaming the root dir of a nested EZ outside the parent EZ is not supported now (the "caveat" we discussed above). Can you elaborate a bit on "renaming a non-nested EZ inside an EZ"?
          Hide
          zhz Zhe Zhang added a comment -

          Forgot to add to the comment that the patch includes the new test from HDFS-9688 patch with necessary modifications. I think the test can be a part of this patch, as well as a standalone unit test for 2.8/2.7/2.6. The purpose of the standalone unit test is to guarantee downgrade support.

          Show
          zhz Zhe Zhang added a comment - Forgot to add to the comment that the patch includes the new test from HDFS-9688 patch with necessary modifications. I think the test can be a part of this patch, as well as a standalone unit test for 2.8/2.7/2.6. The purpose of the standalone unit test is to guarantee downgrade support.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 0m 33s Maven dependency ordering for branch
          +1 mvninstall 6m 53s trunk passed
          +1 compile 1m 17s trunk passed with JDK v1.8.0_72
          +1 compile 1m 19s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 27s trunk passed
          +1 mvnsite 1m 25s trunk passed
          +1 mvneclipse 0m 26s trunk passed
          +1 findbugs 3m 35s trunk passed
          +1 javadoc 1m 28s trunk passed with JDK v1.8.0_72
          +1 javadoc 2m 8s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 17s Maven dependency ordering for patch
          +1 mvninstall 1m 14s the patch passed
          +1 compile 1m 12s the patch passed with JDK v1.8.0_72
          +1 javac 1m 12s the patch passed
          +1 compile 1m 19s the patch passed with JDK v1.7.0_95
          +1 javac 1m 19s the patch passed
          +1 checkstyle 0m 24s the patch passed
          +1 mvnsite 1m 23s the patch passed
          +1 mvneclipse 0m 21s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 3m 59s the patch passed
          +1 javadoc 1m 21s the patch passed with JDK v1.8.0_72
          +1 javadoc 2m 6s the patch passed with JDK v1.7.0_95
          +1 unit 0m 48s hadoop-hdfs-client in the patch passed with JDK v1.8.0_72.
          -1 unit 52m 49s hadoop-hdfs in the patch failed with JDK v1.8.0_72.
          +1 unit 0m 56s hadoop-hdfs-client in the patch passed with JDK v1.7.0_95.
          -1 unit 52m 32s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 21s Patch does not generate ASF License warnings.
          143m 7s



          Reason Tests
          JDK v1.8.0_72 Failed junit tests hadoop.hdfs.TestEncryptionZones
            hadoop.hdfs.TestFileAppend
            hadoop.cli.TestCryptoAdminCLI
            hadoop.hdfs.TestEncryptionZonesWithKMS
          JDK v1.7.0_95 Failed junit tests hadoop.hdfs.TestReconstructStripedFile
            hadoop.hdfs.server.namenode.TestDecommissioningStatus
            hadoop.hdfs.TestEncryptionZones
            hadoop.cli.TestCryptoAdminCLI
            hadoop.hdfs.TestEncryptionZonesWithKMS



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12786318/HDFS-9244.05.patch
          JIRA Issue HDFS-9244
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux b7cc31af4488 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 1bcfab8
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14388/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_72.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14388/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14388/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14388/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14388/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
          Max memory used 76MB
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14388/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 33s Maven dependency ordering for branch +1 mvninstall 6m 53s trunk passed +1 compile 1m 17s trunk passed with JDK v1.8.0_72 +1 compile 1m 19s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 27s trunk passed +1 mvnsite 1m 25s trunk passed +1 mvneclipse 0m 26s trunk passed +1 findbugs 3m 35s trunk passed +1 javadoc 1m 28s trunk passed with JDK v1.8.0_72 +1 javadoc 2m 8s trunk passed with JDK v1.7.0_95 0 mvndep 0m 17s Maven dependency ordering for patch +1 mvninstall 1m 14s the patch passed +1 compile 1m 12s the patch passed with JDK v1.8.0_72 +1 javac 1m 12s the patch passed +1 compile 1m 19s the patch passed with JDK v1.7.0_95 +1 javac 1m 19s the patch passed +1 checkstyle 0m 24s the patch passed +1 mvnsite 1m 23s the patch passed +1 mvneclipse 0m 21s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 3m 59s the patch passed +1 javadoc 1m 21s the patch passed with JDK v1.8.0_72 +1 javadoc 2m 6s the patch passed with JDK v1.7.0_95 +1 unit 0m 48s hadoop-hdfs-client in the patch passed with JDK v1.8.0_72. -1 unit 52m 49s hadoop-hdfs in the patch failed with JDK v1.8.0_72. +1 unit 0m 56s hadoop-hdfs-client in the patch passed with JDK v1.7.0_95. -1 unit 52m 32s hadoop-hdfs in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 21s Patch does not generate ASF License warnings. 143m 7s Reason Tests JDK v1.8.0_72 Failed junit tests hadoop.hdfs.TestEncryptionZones   hadoop.hdfs.TestFileAppend   hadoop.cli.TestCryptoAdminCLI   hadoop.hdfs.TestEncryptionZonesWithKMS JDK v1.7.0_95 Failed junit tests hadoop.hdfs.TestReconstructStripedFile   hadoop.hdfs.server.namenode.TestDecommissioningStatus   hadoop.hdfs.TestEncryptionZones   hadoop.cli.TestCryptoAdminCLI   hadoop.hdfs.TestEncryptionZonesWithKMS Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12786318/HDFS-9244.05.patch JIRA Issue HDFS-9244 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux b7cc31af4488 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 1bcfab8 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/14388/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_72.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14388/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14388/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14388/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14388/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14388/console This message was automatically generated.
          Hide
          zhz Zhe Zhang added a comment -

          Whoops, shouldn't have changed the error message in the FileNotFoundException. Uploading v06 patch to fix the failed unit tests.

          Show
          zhz Zhe Zhang added a comment - Whoops, shouldn't have changed the error message in the FileNotFoundException . Uploading v06 patch to fix the failed unit tests.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 11s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 0m 27s Maven dependency ordering for branch
          +1 mvninstall 6m 59s trunk passed
          +1 compile 1m 26s trunk passed with JDK v1.8.0_72
          +1 compile 1m 26s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 27s trunk passed
          +1 mvnsite 1m 28s trunk passed
          +1 mvneclipse 0m 27s trunk passed
          +1 findbugs 3m 44s trunk passed
          +1 javadoc 1m 29s trunk passed with JDK v1.8.0_72
          +1 javadoc 2m 16s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 25s Maven dependency ordering for patch
          +1 mvninstall 1m 22s the patch passed
          +1 compile 1m 21s the patch passed with JDK v1.8.0_72
          +1 javac 1m 21s the patch passed
          +1 compile 1m 20s the patch passed with JDK v1.7.0_95
          +1 javac 1m 20s the patch passed
          +1 checkstyle 0m 26s the patch passed
          +1 mvnsite 1m 26s the patch passed
          +1 mvneclipse 0m 23s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 4m 16s the patch passed
          +1 javadoc 1m 28s the patch passed with JDK v1.8.0_72
          +1 javadoc 2m 16s the patch passed with JDK v1.7.0_95
          +1 unit 0m 55s hadoop-hdfs-client in the patch passed with JDK v1.8.0_72.
          -1 unit 54m 27s hadoop-hdfs in the patch failed with JDK v1.8.0_72.
          +1 unit 0m 56s hadoop-hdfs-client in the patch passed with JDK v1.7.0_95.
          -1 unit 51m 15s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 20s Patch does not generate ASF License warnings.
          145m 16s



          Reason Tests
          JDK v1.8.0_72 Failed junit tests hadoop.hdfs.server.datanode.TestBlockScanner
          JDK v1.7.0_95 Failed junit tests hadoop.hdfs.TestAclsEndToEnd



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12786350/HDFS-9244.06.patch
          JIRA Issue HDFS-9244
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 75af77996de8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 1bcfab8
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14391/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_72.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14391/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14391/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14391/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14391/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
          Max memory used 77MB
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14391/console
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 27s Maven dependency ordering for branch +1 mvninstall 6m 59s trunk passed +1 compile 1m 26s trunk passed with JDK v1.8.0_72 +1 compile 1m 26s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 27s trunk passed +1 mvnsite 1m 28s trunk passed +1 mvneclipse 0m 27s trunk passed +1 findbugs 3m 44s trunk passed +1 javadoc 1m 29s trunk passed with JDK v1.8.0_72 +1 javadoc 2m 16s trunk passed with JDK v1.7.0_95 0 mvndep 0m 25s Maven dependency ordering for patch +1 mvninstall 1m 22s the patch passed +1 compile 1m 21s the patch passed with JDK v1.8.0_72 +1 javac 1m 21s the patch passed +1 compile 1m 20s the patch passed with JDK v1.7.0_95 +1 javac 1m 20s the patch passed +1 checkstyle 0m 26s the patch passed +1 mvnsite 1m 26s the patch passed +1 mvneclipse 0m 23s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 4m 16s the patch passed +1 javadoc 1m 28s the patch passed with JDK v1.8.0_72 +1 javadoc 2m 16s the patch passed with JDK v1.7.0_95 +1 unit 0m 55s hadoop-hdfs-client in the patch passed with JDK v1.8.0_72. -1 unit 54m 27s hadoop-hdfs in the patch failed with JDK v1.8.0_72. +1 unit 0m 56s hadoop-hdfs-client in the patch passed with JDK v1.7.0_95. -1 unit 51m 15s hadoop-hdfs in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 20s Patch does not generate ASF License warnings. 145m 16s Reason Tests JDK v1.8.0_72 Failed junit tests hadoop.hdfs.server.datanode.TestBlockScanner JDK v1.7.0_95 Failed junit tests hadoop.hdfs.TestAclsEndToEnd Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12786350/HDFS-9244.06.patch JIRA Issue HDFS-9244 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 75af77996de8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 1bcfab8 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/14391/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_72.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14391/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14391/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14391/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14391/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Max memory used 77MB Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14391/console Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          zhz Zhe Zhang added a comment -

          Reported failures are unrelated and pass locally.

          Show
          zhz Zhe Zhang added a comment - Reported failures are unrelated and pass locally.
          Hide
          andrew.wang Andrew Wang added a comment -

          I forgot about the caveat, but we can still assert the behavior that renaming /ez1/ez2 to /ez2 does not work, and /ez2 to /ez1/ez2 does not work. Otherwise I'm +1 pending, nice work here Zhe Zhang.

          Show
          andrew.wang Andrew Wang added a comment - I forgot about the caveat, but we can still assert the behavior that renaming /ez1/ez2 to /ez2 does not work, and /ez2 to /ez1/ez2 does not work. Otherwise I'm +1 pending, nice work here Zhe Zhang .
          Hide
          zhz Zhe Zhang added a comment -

          Thanks Andrew for the review!

          // Verify that the nested EZ cannot be moved out of the top EZ.
          

          This section already verifies renaming /ez1/ez2 to /ez2. I just added another section under it to verify the other case you mentioned.

          Show
          zhz Zhe Zhang added a comment - Thanks Andrew for the review! // Verify that the nested EZ cannot be moved out of the top EZ. This section already verifies renaming /ez1/ez2 to /ez2. I just added another section under it to verify the other case you mentioned.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 25s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 0m 28s Maven dependency ordering for branch
          +1 mvninstall 8m 8s trunk passed
          +1 compile 1m 51s trunk passed with JDK v1.8.0_66
          +1 compile 1m 36s trunk passed with JDK v1.7.0_91
          +1 checkstyle 0m 28s trunk passed
          +1 mvnsite 1m 33s trunk passed
          +1 mvneclipse 0m 25s trunk passed
          +1 findbugs 3m 53s trunk passed
          +1 javadoc 1m 46s trunk passed with JDK v1.8.0_66
          +1 javadoc 2m 34s trunk passed with JDK v1.7.0_91
          0 mvndep 0m 18s Maven dependency ordering for patch
          +1 mvninstall 1m 31s the patch passed
          +1 compile 1m 57s the patch passed with JDK v1.8.0_66
          +1 javac 1m 57s the patch passed
          +1 compile 1m 39s the patch passed with JDK v1.7.0_91
          +1 javac 1m 39s the patch passed
          +1 checkstyle 0m 26s the patch passed
          +1 mvnsite 1m 35s the patch passed
          +1 mvneclipse 0m 22s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 4m 20s the patch passed
          +1 javadoc 1m 38s the patch passed with JDK v1.8.0_66
          +1 javadoc 2m 20s the patch passed with JDK v1.7.0_91
          +1 unit 1m 2s hadoop-hdfs-client in the patch passed with JDK v1.8.0_66.
          -1 unit 91m 1s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
          +1 unit 1m 13s hadoop-hdfs-client in the patch passed with JDK v1.7.0_91.
          -1 unit 84m 31s hadoop-hdfs in the patch failed with JDK v1.7.0_91.
          +1 asflicense 0m 21s Patch does not generate ASF License warnings.
          220m 7s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.hdfs.server.datanode.TestBlockScanner
            hadoop.hdfs.server.blockmanagement.TestBlockManager
            hadoop.hdfs.security.TestDelegationTokenForProxyUser
            hadoop.hdfs.TestFileAppend
            hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
            hadoop.hdfs.server.datanode.fsdataset.impl.TestScrLazyPersistFiles
            hadoop.hdfs.server.datanode.TestDirectoryScanner
          JDK v1.7.0_91 Failed junit tests hadoop.hdfs.server.datanode.TestBlockScanner
            hadoop.hdfs.shortcircuit.TestShortCircuitCache
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
            hadoop.hdfs.server.datanode.TestBlockReplacement
            hadoop.hdfs.server.datanode.TestDirectoryScanner



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12786547/HDFS-9244.07.patch
          JIRA Issue HDFS-9244
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux e01951dc8ee1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 9086dd5
          Default Java 1.7.0_91
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14411/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14411/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14411/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14411/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14411/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
          Max memory used 77MB
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14411/console
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 25s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 28s Maven dependency ordering for branch +1 mvninstall 8m 8s trunk passed +1 compile 1m 51s trunk passed with JDK v1.8.0_66 +1 compile 1m 36s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 28s trunk passed +1 mvnsite 1m 33s trunk passed +1 mvneclipse 0m 25s trunk passed +1 findbugs 3m 53s trunk passed +1 javadoc 1m 46s trunk passed with JDK v1.8.0_66 +1 javadoc 2m 34s trunk passed with JDK v1.7.0_91 0 mvndep 0m 18s Maven dependency ordering for patch +1 mvninstall 1m 31s the patch passed +1 compile 1m 57s the patch passed with JDK v1.8.0_66 +1 javac 1m 57s the patch passed +1 compile 1m 39s the patch passed with JDK v1.7.0_91 +1 javac 1m 39s the patch passed +1 checkstyle 0m 26s the patch passed +1 mvnsite 1m 35s the patch passed +1 mvneclipse 0m 22s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 4m 20s the patch passed +1 javadoc 1m 38s the patch passed with JDK v1.8.0_66 +1 javadoc 2m 20s the patch passed with JDK v1.7.0_91 +1 unit 1m 2s hadoop-hdfs-client in the patch passed with JDK v1.8.0_66. -1 unit 91m 1s hadoop-hdfs in the patch failed with JDK v1.8.0_66. +1 unit 1m 13s hadoop-hdfs-client in the patch passed with JDK v1.7.0_91. -1 unit 84m 31s hadoop-hdfs in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 21s Patch does not generate ASF License warnings. 220m 7s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.hdfs.server.datanode.TestBlockScanner   hadoop.hdfs.server.blockmanagement.TestBlockManager   hadoop.hdfs.security.TestDelegationTokenForProxyUser   hadoop.hdfs.TestFileAppend   hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints   hadoop.hdfs.server.datanode.fsdataset.impl.TestScrLazyPersistFiles   hadoop.hdfs.server.datanode.TestDirectoryScanner JDK v1.7.0_91 Failed junit tests hadoop.hdfs.server.datanode.TestBlockScanner   hadoop.hdfs.shortcircuit.TestShortCircuitCache   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.server.datanode.TestBlockReplacement   hadoop.hdfs.server.datanode.TestDirectoryScanner Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12786547/HDFS-9244.07.patch JIRA Issue HDFS-9244 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux e01951dc8ee1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 9086dd5 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/14411/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14411/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14411/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14411/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14411/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Max memory used 77MB Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14411/console Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          zhz Zhe Zhang added a comment -

          The reported failures are unrelated to the patch (none involves EZs), and pass locally.

          Show
          zhz Zhe Zhang added a comment - The reported failures are unrelated to the patch (none involves EZs), and pass locally.
          Hide
          zhz Zhe Zhang added a comment -

          Committed to trunk, branch-2, and branch-2.8 based on Andrew's +1. Thanks Andrew and Xiaoyu for the helpful reviews!

          Show
          zhz Zhe Zhang added a comment - Committed to trunk, branch-2, and branch-2.8 based on Andrew's +1. Thanks Andrew and Xiaoyu for the helpful reviews!
          Hide
          zhz Zhe Zhang added a comment -

          Also thanks Nicholas, Arpit, and Sanjay for reviewing the design.

          Show
          zhz Zhe Zhang added a comment - Also thanks Nicholas, Arpit, and Sanjay for reviewing the design.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #9264 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9264/)
          HDFS-9244. Support nested encryption zones. (zhezhang: rev dbe49c1bd6f62f04cf4290795b81a66fbd41d44c)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EncryptionZoneManager.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testCryptoConf.xml
          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNestedEncryptionZones.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptionZones.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9264 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9264/ ) HDFS-9244 . Support nested encryption zones. (zhezhang: rev dbe49c1bd6f62f04cf4290795b81a66fbd41d44c) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EncryptionZoneManager.java hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testCryptoConf.xml hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNestedEncryptionZones.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptionZones.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

            People

            • Assignee:
              zhz Zhe Zhang
              Reporter:
              xyao Xiaoyu Yao
            • Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development