Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10883

`getTrashRoot`'s behavior is not consistent in DFS after enabling EZ.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha2
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      If root path / is an encryption zone, the old DistributedFileSystem#getTrashRoot(new Path("/")) returns
      /user/$USER/.Trash
      which is a wrong behavior. The correct value should be
      /.Trash/$USER
      Show
      If root path / is an encryption zone, the old DistributedFileSystem#getTrashRoot(new Path("/")) returns /user/$USER/.Trash which is a wrong behavior. The correct value should be /.Trash/$USER

      Description

      Let's say root path ("/") is the encryption zone, and there is a file called "/test" in root path.

      dfs.getTrashRoot(new Path("/"))
      

      returns "/user/$USER/.Trash",
      while

      dfs.getTrashRoot(new Path("/test"))
      

      returns "/.Trash/$USER".
      The first behavior is not correct. Since root path is the encryption zone, it's more reasonable to return "/.Trash/$USER" no matter what the path is.

      1. HDFS-10883-test-case.txt
        2 kB
        Yuanbo Liu
      2. HDFS-10883.004.patch
        17 kB
        Yuanbo Liu
      3. HDFS-10883.003.patch
        16 kB
        Yuanbo Liu
      4. HDFS-10883.002.patch
        16 kB
        Yuanbo Liu
      5. HDFS-10883.001.patch
        3 kB
        Yuanbo Liu

        Issue Links

          Activity

          Hide
          yuanbo Yuanbo Liu added a comment -

          Thanks for your comment. This is my first time using release note. You and Xiao are really helpful, thanks a lot.

          Show
          yuanbo Yuanbo Liu added a comment - Thanks for your comment. This is my first time using release note. You and Xiao are really helpful, thanks a lot.
          Hide
          andrew.wang Andrew Wang added a comment -

          One more FYI, the Release Notes field is parsed as either plaintext, or as markdown if you start it with "<!-- markdown -->" as the first line. Thanks Yuanbo

          Show
          andrew.wang Andrew Wang added a comment - One more FYI, the Release Notes field is parsed as either plaintext, or as markdown if you start it with "<!-- markdown -->" as the first line. Thanks Yuanbo
          Hide
          yuanbo Yuanbo Liu added a comment -

          Xiao Chen Thanks for your reminder.

          Show
          yuanbo Yuanbo Liu added a comment - Xiao Chen Thanks for your reminder.
          Hide
          xiaochen Xiao Chen added a comment -

          Hi Yuanbo Liu,
          Thanks for the contribution.
          Regarding release notes: if you click 'Edit', you'll see a ‘Release Notes' field.

          Show
          xiaochen Xiao Chen added a comment - Hi Yuanbo Liu , Thanks for the contribution. Regarding release notes: if you click 'Edit', you'll see a ‘Release Notes' field.
          Hide
          yuanbo Yuanbo Liu added a comment - - edited

          Andrew Wang Sure, I'll update the description. Thanks for your commit.

          Show
          yuanbo Yuanbo Liu added a comment - - edited Andrew Wang Sure, I'll update the description. Thanks for your commit.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10613 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10613/)
          HDFS-10883. 's behavior is not consistent in DFS after enabling EZ. (wang: rev 0007360c3344b3485fa17de0fd2015a628de947c)

          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptionZones.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNestedEncryptionZones.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/TransparentEncryption.md
          • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10613 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10613/ ) HDFS-10883 . 's behavior is not consistent in DFS after enabling EZ. (wang: rev 0007360c3344b3485fa17de0fd2015a628de947c) (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptionZones.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNestedEncryptionZones.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/TransparentEncryption.md (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
          Hide
          andrew.wang Andrew Wang added a comment -

          Thanks for working on this Yuanbo, I've committed the patch to trunk, branch-2, branch-2.8.

          Do you mind also editing the JIRA to add a release note? We should help end users understand both the old incorrect behavior and the new correct behavior.

          Show
          andrew.wang Andrew Wang added a comment - Thanks for working on this Yuanbo, I've committed the patch to trunk, branch-2, branch-2.8. Do you mind also editing the JIRA to add a release note? We should help end users understand both the old incorrect behavior and the new correct behavior.
          Hide
          yuanbo Yuanbo Liu added a comment - - edited

          I've tested these failures locally. They seem not related to my code change.
          Establish another Jenkins test.

          Show
          yuanbo Yuanbo Liu added a comment - - edited I've tested these failures locally. They seem not related to my code change. Establish another Jenkins test.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 23s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          0 mvndep 0m 30s Maven dependency ordering for branch
          +1 mvninstall 8m 47s trunk passed
          +1 compile 1m 52s trunk passed
          +1 checkstyle 0m 37s trunk passed
          +1 mvnsite 1m 51s trunk passed
          +1 mvneclipse 0m 31s trunk passed
          +1 findbugs 3m 49s trunk passed
          +1 javadoc 1m 9s trunk passed
          0 mvndep 0m 9s Maven dependency ordering for patch
          +1 mvninstall 1m 38s the patch passed
          +1 compile 1m 33s the patch passed
          +1 javac 1m 33s the patch passed
          +1 checkstyle 0m 29s the patch passed
          +1 mvnsite 1m 34s the patch passed
          +1 mvneclipse 0m 24s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 3m 50s the patch passed
          +1 javadoc 1m 11s the patch passed
          +1 unit 1m 7s hadoop-hdfs-client in the patch passed.
          -1 unit 97m 25s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 28s The patch does not generate ASF License warnings.
          130m 59s



          Reason Tests
          Failed junit tests hadoop.hdfs.TestPersistBlocks
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
          Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue HDFS-10883
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833295/HDFS-10883.004.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux f956bbbbc7d9 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / cf3f43e
          Default Java 1.8.0_101
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/17149/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17149/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17149/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 23s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 30s Maven dependency ordering for branch +1 mvninstall 8m 47s trunk passed +1 compile 1m 52s trunk passed +1 checkstyle 0m 37s trunk passed +1 mvnsite 1m 51s trunk passed +1 mvneclipse 0m 31s trunk passed +1 findbugs 3m 49s trunk passed +1 javadoc 1m 9s trunk passed 0 mvndep 0m 9s Maven dependency ordering for patch +1 mvninstall 1m 38s the patch passed +1 compile 1m 33s the patch passed +1 javac 1m 33s the patch passed +1 checkstyle 0m 29s the patch passed +1 mvnsite 1m 34s the patch passed +1 mvneclipse 0m 24s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 50s the patch passed +1 javadoc 1m 11s the patch passed +1 unit 1m 7s hadoop-hdfs-client in the patch passed. -1 unit 97m 25s hadoop-hdfs in the patch failed. +1 asflicense 0m 28s The patch does not generate ASF License warnings. 130m 59s Reason Tests Failed junit tests hadoop.hdfs.TestPersistBlocks   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2 Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-10883 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833295/HDFS-10883.004.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux f956bbbbc7d9 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / cf3f43e Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/17149/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17149/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17149/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          yuanbo Yuanbo Liu added a comment -

          Andrew Wang
          Thanks for your review, upload v4 patch to address your comment.

          Show
          yuanbo Yuanbo Liu added a comment - Andrew Wang Thanks for your review, upload v4 patch to address your comment.
          Hide
          andrew.wang Andrew Wang added a comment -

          Thanks for working on this Yuanbo Liu, just one nit:

          • testNestedEncryptionZones, we can just let the exception bubble rather than catching and doing an explicit "fail". You can put the fail message as a comment.

          Otherwise looks good, thanks for working on this!

          Show
          andrew.wang Andrew Wang added a comment - Thanks for working on this Yuanbo Liu , just one nit: testNestedEncryptionZones, we can just let the exception bubble rather than catching and doing an explicit "fail". You can put the fail message as a comment. Otherwise looks good, thanks for working on this!
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          0 mvndep 0m 30s Maven dependency ordering for branch
          +1 mvninstall 8m 42s trunk passed
          +1 compile 1m 48s trunk passed
          +1 checkstyle 0m 34s trunk passed
          +1 mvnsite 1m 36s trunk passed
          +1 mvneclipse 0m 27s trunk passed
          +1 findbugs 3m 34s trunk passed
          +1 javadoc 1m 25s trunk passed
          0 mvndep 0m 7s Maven dependency ordering for patch
          +1 mvninstall 1m 33s the patch passed
          +1 compile 1m 41s the patch passed
          +1 javac 1m 41s the patch passed
          +1 checkstyle 0m 30s the patch passed
          +1 mvnsite 1m 21s the patch passed
          +1 mvneclipse 0m 19s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 3m 14s the patch passed
          +1 javadoc 1m 10s the patch passed
          +1 unit 0m 53s hadoop-hdfs-client in the patch passed.
          -1 unit 63m 52s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 23s The patch does not generate ASF License warnings.
          95m 23s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue HDFS-10883
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830618/HDFS-10883.003.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux c4e56ecbaf7a 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 6437ba1
          Default Java 1.8.0_101
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/16897/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16897/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16897/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 30s Maven dependency ordering for branch +1 mvninstall 8m 42s trunk passed +1 compile 1m 48s trunk passed +1 checkstyle 0m 34s trunk passed +1 mvnsite 1m 36s trunk passed +1 mvneclipse 0m 27s trunk passed +1 findbugs 3m 34s trunk passed +1 javadoc 1m 25s trunk passed 0 mvndep 0m 7s Maven dependency ordering for patch +1 mvninstall 1m 33s the patch passed +1 compile 1m 41s the patch passed +1 javac 1m 41s the patch passed +1 checkstyle 0m 30s the patch passed +1 mvnsite 1m 21s the patch passed +1 mvneclipse 0m 19s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 14s the patch passed +1 javadoc 1m 10s the patch passed +1 unit 0m 53s hadoop-hdfs-client in the patch passed. -1 unit 63m 52s hadoop-hdfs in the patch failed. +1 asflicense 0m 23s The patch does not generate ASF License warnings. 95m 23s Reason Tests Failed junit tests hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-10883 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830618/HDFS-10883.003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c4e56ecbaf7a 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6437ba1 Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/16897/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16897/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16897/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          yuanbo Yuanbo Liu added a comment -

          Upload v3 patch to address test failure.

          Show
          yuanbo Yuanbo Liu added a comment - Upload v3 patch to address test failure.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          0 mvndep 0m 29s Maven dependency ordering for branch
          +1 mvninstall 8m 37s trunk passed
          +1 compile 1m 38s trunk passed
          +1 checkstyle 0m 32s trunk passed
          +1 mvnsite 1m 35s trunk passed
          +1 mvneclipse 0m 26s trunk passed
          +1 findbugs 3m 28s trunk passed
          +1 javadoc 1m 15s trunk passed
          0 mvndep 0m 6s Maven dependency ordering for patch
          +1 mvninstall 1m 20s the patch passed
          +1 compile 1m 34s the patch passed
          +1 javac 1m 34s the patch passed
          +1 checkstyle 0m 31s the patch passed
          +1 mvnsite 1m 21s the patch passed
          +1 mvneclipse 0m 19s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 3m 16s the patch passed
          +1 javadoc 1m 11s the patch passed
          +1 unit 0m 54s hadoop-hdfs-client in the patch passed.
          -1 unit 58m 14s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 23s The patch does not generate ASF License warnings.
          88m 49s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.namenode.TestNestedEncryptionZones



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue HDFS-10883
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830486/HDFS-10883.002.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux fe0780ab211b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / e17a497
          Default Java 1.8.0_101
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/16880/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16880/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16880/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 29s Maven dependency ordering for branch +1 mvninstall 8m 37s trunk passed +1 compile 1m 38s trunk passed +1 checkstyle 0m 32s trunk passed +1 mvnsite 1m 35s trunk passed +1 mvneclipse 0m 26s trunk passed +1 findbugs 3m 28s trunk passed +1 javadoc 1m 15s trunk passed 0 mvndep 0m 6s Maven dependency ordering for patch +1 mvninstall 1m 20s the patch passed +1 compile 1m 34s the patch passed +1 javac 1m 34s the patch passed +1 checkstyle 0m 31s the patch passed +1 mvnsite 1m 21s the patch passed +1 mvneclipse 0m 19s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 16s the patch passed +1 javadoc 1m 11s the patch passed +1 unit 0m 54s hadoop-hdfs-client in the patch passed. -1 unit 58m 14s hadoop-hdfs in the patch failed. +1 asflicense 0m 23s The patch does not generate ASF License warnings. 88m 49s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.TestNestedEncryptionZones Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-10883 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830486/HDFS-10883.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux fe0780ab211b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e17a497 Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/16880/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16880/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16880/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          yuanbo Yuanbo Liu added a comment -

          Uploaded v2 patch for this issue. Andrew Wang/Xiaoyu Yao/Wei-Chiu Chuang, would you please take a look if you have time? Thanks in advance.

          Show
          yuanbo Yuanbo Liu added a comment - Uploaded v2 patch for this issue. Andrew Wang / Xiaoyu Yao / Wei-Chiu Chuang , would you please take a look if you have time? Thanks in advance.
          Hide
          xyao Xiaoyu Yao added a comment -

          I think the second behavior is more consistent, because of the case Weichiu mentioned with nested EZs. That is, getTrashRoot("/") should return /.Trash/USER. I don't know why there is a special case.

          I agree 2nd behavior is preferred.

          How about adding an subsection here: TransparentEncryption.md#Architecture and name it as "operation with a nested encryption zone".

          HDFS-9644 is the document JIRA added for nested encryption zone support. We could also improve the newly added "Rename_and_Trash_considerations" section to document not only the case for file rename but also the directory rename, especially the root of the encryption zone cases.

          Show
          xyao Xiaoyu Yao added a comment - I think the second behavior is more consistent, because of the case Weichiu mentioned with nested EZs. That is, getTrashRoot("/") should return /.Trash/USER. I don't know why there is a special case. I agree 2nd behavior is preferred. How about adding an subsection here: TransparentEncryption.md#Architecture and name it as "operation with a nested encryption zone". HDFS-9644 is the document JIRA added for nested encryption zone support. We could also improve the newly added "Rename_and_Trash_considerations" section to document not only the case for file rename but also the directory rename, especially the root of the encryption zone cases.
          Hide
          yuanbo Yuanbo Liu added a comment -

          Xiaoyu Yao Thanks for your comments.

          Also, we need to update the document regarding nested encryption...

          How about adding an subsection here: TransparentEncryption.md#Architecture and name it as "operation with a nested encryption zone".

          Show
          yuanbo Yuanbo Liu added a comment - Xiaoyu Yao Thanks for your comments. Also, we need to update the document regarding nested encryption... How about adding an subsection here: TransparentEncryption.md#Architecture and name it as "operation with a nested encryption zone".
          Hide
          yuanbo Yuanbo Liu added a comment - - edited

          Andrew Wang Sure, I can work on it if you don't mind.

          I think the second behavior is more consistent...

          Wei-Chiu Chuang's comment reminds me that things're gonna be complicated if we take getTrashRoot("/") as a special case. I will change my description.

          Show
          yuanbo Yuanbo Liu added a comment - - edited Andrew Wang Sure, I can work on it if you don't mind. I think the second behavior is more consistent... Wei-Chiu Chuang 's comment reminds me that things're gonna be complicated if we take getTrashRoot("/") as a special case. I will change my description.
          Hide
          andrew.wang Andrew Wang added a comment -

          I think the second behavior is more consistent, because of the case Weichiu mentioned with nested EZs. That is, getTrashRoot("/") should return /.Trash/USER. I don't know why there is a special case.

          I tried making this change (removing the isRoot special case), and it looks like we missed a test case asserting this behavior in TestEncryptionZones and TestNestedEncryptionZones.

          Yuanbo Liu do you want to work on this? Else I can provide a patch.

          Show
          andrew.wang Andrew Wang added a comment - I think the second behavior is more consistent, because of the case Weichiu mentioned with nested EZs. That is, getTrashRoot("/") should return /.Trash/USER . I don't know why there is a special case. I tried making this change (removing the isRoot special case), and it looks like we missed a test case asserting this behavior in TestEncryptionZones and TestNestedEncryptionZones. Yuanbo Liu do you want to work on this? Else I can provide a patch.
          Hide
          xyao Xiaoyu Yao added a comment -

          Yuanbo Liu, thanks for reporting the issue and posting the patches.

          The second behavior is not correct. Since root path is the encryption zone, which means all files/directories in DFS are encrypted, it's more reasonable to return "/user/$USER/.Trash" no matter what the path is.

          Taking a looking at the history of changes in getTrashRoot API, the special case when encryption zone is created on "/" is added by HDFS-9244 to support nested encryption zone. Before that, the API always keeps the trash of encryption zone always under the ".Trash" directory of the encryption zone's root. Zhe Zhang/Andrew Wang, can you provide more context why this special case is needed?

          Also, we need to update the document regarding nested encryption support and its impact to rename and Trash support.

          Show
          xyao Xiaoyu Yao added a comment - Yuanbo Liu , thanks for reporting the issue and posting the patches. The second behavior is not correct. Since root path is the encryption zone, which means all files/directories in DFS are encrypted, it's more reasonable to return "/user/$USER/.Trash" no matter what the path is. Taking a looking at the history of changes in getTrashRoot API, the special case when encryption zone is created on "/" is added by HDFS-9244 to support nested encryption zone. Before that, the API always keeps the trash of encryption zone always under the ".Trash" directory of the encryption zone's root. Zhe Zhang / Andrew Wang , can you provide more context why this special case is needed? Also, we need to update the document regarding nested encryption support and its impact to rename and Trash support.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          The other corner case is /user as an encryption zone.

          Show
          jojochuang Wei-Chiu Chuang added a comment - The other corner case is /user as an encryption zone.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 12s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          0 mvndep 0m 9s Maven dependency ordering for branch
          +1 mvninstall 8m 56s trunk passed
          +1 compile 1m 35s trunk passed
          +1 checkstyle 0m 32s trunk passed
          +1 mvnsite 1m 42s trunk passed
          +1 mvneclipse 0m 28s trunk passed
          +1 findbugs 3m 51s trunk passed
          +1 javadoc 1m 31s trunk passed
          0 mvndep 0m 8s Maven dependency ordering for patch
          +1 mvninstall 1m 43s the patch passed
          +1 compile 1m 26s the patch passed
          +1 javac 1m 26s the patch passed
          +1 checkstyle 0m 28s the patch passed
          +1 mvnsite 1m 25s the patch passed
          +1 mvneclipse 0m 20s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 3m 23s the patch passed
          +1 javadoc 1m 12s the patch passed
          +1 unit 0m 54s hadoop-hdfs-client in the patch passed.
          -1 unit 62m 34s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 21s The patch does not generate ASF License warnings.
          94m 20s



          Reason Tests
          Failed junit tests hadoop.hdfs.TestDFSShell



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue HDFS-10883
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829531/HDFS-10883.001.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux b8cd7d847330 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 964e546
          Default Java 1.8.0_101
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/16823/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16823/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16823/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 12s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 9s Maven dependency ordering for branch +1 mvninstall 8m 56s trunk passed +1 compile 1m 35s trunk passed +1 checkstyle 0m 32s trunk passed +1 mvnsite 1m 42s trunk passed +1 mvneclipse 0m 28s trunk passed +1 findbugs 3m 51s trunk passed +1 javadoc 1m 31s trunk passed 0 mvndep 0m 8s Maven dependency ordering for patch +1 mvninstall 1m 43s the patch passed +1 compile 1m 26s the patch passed +1 javac 1m 26s the patch passed +1 checkstyle 0m 28s the patch passed +1 mvnsite 1m 25s the patch passed +1 mvneclipse 0m 20s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 23s the patch passed +1 javadoc 1m 12s the patch passed +1 unit 0m 54s hadoop-hdfs-client in the patch passed. -1 unit 62m 34s hadoop-hdfs in the patch failed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 94m 20s Reason Tests Failed junit tests hadoop.hdfs.TestDFSShell Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-10883 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829531/HDFS-10883.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux b8cd7d847330 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 964e546 Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/16823/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16823/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16823/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          yuanbo Yuanbo Liu added a comment -

          upload v1 patch for this issue.

          Show
          yuanbo Yuanbo Liu added a comment - upload v1 patch for this issue.
          Hide
          yuanbo Yuanbo Liu added a comment -

          Xiao Chen After discussing with you in HDFS-10756, I tested the behavior of getTrashRoot in DFS and found this issue, I've uploaded a test case file in this jira and hope to get your thoughts.

          Show
          yuanbo Yuanbo Liu added a comment - Xiao Chen After discussing with you in HDFS-10756 , I tested the behavior of getTrashRoot in DFS and found this issue, I've uploaded a test case file in this jira and hope to get your thoughts.

            People

            • Assignee:
              yuanbo Yuanbo Liu
              Reporter:
              yuanbo Yuanbo Liu
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development