Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11100

Recursively deleting file protected by sticky bit should fail

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 3.0.0-alpha4
    • Component/s: fs
    • Labels:
    • Target Version/s:
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      Changed the behavior of removing directories with sticky bits, so that it is closer to what most Unix/Linux users would expect.

      Description

      Recursively deleting a directory that contains files or directories protected by sticky bit should fail but it doesn't in HDFS. In the case below, /tmp/test/sticky_dir/f2 is protected by sticky bit, thus recursive deleting /tmp/test/sticky_dir should fail.

      + hdfs dfs -ls -R /tmp/test
      drwxrwxrwt   - jzhuge supergroup          0 2016-11-03 18:08 /tmp/test/sticky_dir
      -rwxrwxrwx   1 jzhuge supergroup          0 2016-11-03 18:08 /tmp/test/sticky_dir/f2
      
      + sudo -u hadoop hdfs dfs -rm -skipTrash /tmp/test/sticky_dir/f2
      rm: Permission denied by sticky bit: user=hadoop, path="/tmp/test/sticky_dir/f2":jzhuge:supergroup:-rwxrwxrwx, parent="/tmp/test/sticky_dir":jzhuge:supergroup:drwxrwxrwt
      
      + sudo -u hadoop hdfs dfs -rm -r -skipTrash /tmp/test/sticky_dir
      Deleted /tmp/test/sticky_dir
      

      Centos 6.4 behavior:

      $ ls -lR /tmp/test
      /tmp/test: 
      total 4
      drwxrwxrwt 2 systest systest 4096 Nov  3 18:36 sbit
      
      /tmp/test/sbit:
      total 0
      -rw-rw-rw- 1 systest systest 0 Nov  2 13:45 f2
      
      $ sudo -u mapred rm -fr /tmp/test/sbit
      rm: cannot remove `/tmp/test/sbit/f2': Operation not permitted
      
      $ chmod -t /tmp/test/sbit
      $ sudo -u mapred rm -fr /tmp/test/sbit
      
      1. hdfs_cmds
        1 kB
        John Zhuge
      2. HDFS-11100.001.patch
        7 kB
        John Zhuge
      3. HDFS-11100.002.patch
        7 kB
        John Zhuge
      4. HDFS-11100.003.patch
        7 kB
        John Zhuge
      5. HDFS-11100.004.patch
        10 kB
        John Zhuge
      6. HDFS-11100.005.patch
        10 kB
        John Zhuge

        Activity

        Hide
        jzhuge John Zhuge added a comment -

        checkStickyBit is only called for the current inode for the recursive delete. It is /tmp/test/sbit in the example. Maybe we should check sticky bit recursively in the subtree? It would be an incompatible change.

        Show
        jzhuge John Zhuge added a comment - checkStickyBit is only called for the current inode for the recursive delete. It is /tmp/test/sbit in the example. Maybe we should check sticky bit recursively in the subtree? It would be an incompatible change.
        Hide
        jzhuge John Zhuge added a comment -

        Attach hdfs_cmds, a script to reproduce the issue.

        Show
        jzhuge John Zhuge added a comment - Attach hdfs_cmds , a script to reproduce the issue.
        Hide
        jzhuge John Zhuge added a comment -

        Patch 001:

        • Check sticky bit in checkSubAccess
        Show
        jzhuge John Zhuge added a comment - Patch 001: Check sticky bit in checkSubAccess
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 22s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 1m 36s Maven dependency ordering for branch
        +1 mvninstall 6m 44s trunk passed
        +1 compile 11m 16s trunk passed
        +1 checkstyle 1m 36s trunk passed
        +1 mvnsite 2m 6s trunk passed
        +1 mvneclipse 0m 44s trunk passed
        +1 findbugs 3m 18s trunk passed
        +1 javadoc 1m 37s trunk passed
        0 mvndep 0m 14s Maven dependency ordering for patch
        +1 mvninstall 1m 21s the patch passed
        +1 compile 9m 19s the patch passed
        +1 javac 9m 19s the patch passed
        +1 checkstyle 1m 37s the patch passed
        +1 mvnsite 2m 6s the patch passed
        +1 mvneclipse 0m 44s the patch passed
        -1 whitespace 0m 0s The patch has 3 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
        +1 findbugs 3m 35s the patch passed
        +1 javadoc 1m 38s the patch passed
        +1 unit 7m 23s hadoop-common in the patch passed.
        +1 unit 54m 30s hadoop-hdfs in the patch passed.
        +1 asflicense 0m 40s The patch does not generate ASF License warnings.
        113m 48s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:e809691
        JIRA Issue HDFS-11100
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12838332/HDFS-11100.001.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 423e42f128cb 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 86ac1ad
        Default Java 1.8.0_101
        findbugs v3.0.0
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/17502/artifact/patchprocess/whitespace-eol.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17502/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17502/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 22s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 1m 36s Maven dependency ordering for branch +1 mvninstall 6m 44s trunk passed +1 compile 11m 16s trunk passed +1 checkstyle 1m 36s trunk passed +1 mvnsite 2m 6s trunk passed +1 mvneclipse 0m 44s trunk passed +1 findbugs 3m 18s trunk passed +1 javadoc 1m 37s trunk passed 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 1m 21s the patch passed +1 compile 9m 19s the patch passed +1 javac 9m 19s the patch passed +1 checkstyle 1m 37s the patch passed +1 mvnsite 2m 6s the patch passed +1 mvneclipse 0m 44s the patch passed -1 whitespace 0m 0s The patch has 3 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply +1 findbugs 3m 35s the patch passed +1 javadoc 1m 38s the patch passed +1 unit 7m 23s hadoop-common in the patch passed. +1 unit 54m 30s hadoop-hdfs in the patch passed. +1 asflicense 0m 40s The patch does not generate ASF License warnings. 113m 48s Subsystem Report/Notes Docker Image:yetus/hadoop:e809691 JIRA Issue HDFS-11100 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12838332/HDFS-11100.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 423e42f128cb 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 86ac1ad Default Java 1.8.0_101 findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/17502/artifact/patchprocess/whitespace-eol.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17502/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17502/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        jzhuge John Zhuge added a comment -

        Patch 002:

        • Fix whitespace
        Show
        jzhuge John Zhuge added a comment - Patch 002: Fix whitespace
        Hide
        jzhuge John Zhuge added a comment -

        Would like to optimize the code a little.

        Show
        jzhuge John Zhuge added a comment - Would like to optimize the code a little.
        Hide
        jzhuge John Zhuge added a comment -

        Patch 002:

        • Optimization: do not check children for sticky bit protection if the parent does not have the bit set
        Show
        jzhuge John Zhuge added a comment - Patch 002: Optimization: do not check children for sticky bit protection if the parent does not have the bit set
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 19s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 17s Maven dependency ordering for branch
        +1 mvninstall 7m 23s trunk passed
        +1 compile 11m 35s trunk passed
        +1 checkstyle 1m 50s trunk passed
        +1 mvnsite 2m 20s trunk passed
        +1 mvneclipse 0m 46s trunk passed
        +1 findbugs 3m 37s trunk passed
        +1 javadoc 1m 41s trunk passed
        0 mvndep 0m 15s Maven dependency ordering for patch
        +1 mvninstall 1m 36s the patch passed
        +1 compile 10m 55s the patch passed
        +1 javac 10m 55s the patch passed
        +1 checkstyle 1m 42s the patch passed
        +1 mvnsite 2m 22s the patch passed
        +1 mvneclipse 0m 41s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 4m 6s the patch passed
        +1 javadoc 1m 43s the patch passed
        +1 unit 8m 36s hadoop-common in the patch passed.
        +1 unit 79m 20s hadoop-hdfs in the patch passed.
        +1 asflicense 0m 51s The patch does not generate ASF License warnings.
        143m 16s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:e809691
        JIRA Issue HDFS-11100
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12838384/HDFS-11100.002.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux df190ab3d1ce 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / ca68f9c
        Default Java 1.8.0_101
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17507/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17507/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 17s Maven dependency ordering for branch +1 mvninstall 7m 23s trunk passed +1 compile 11m 35s trunk passed +1 checkstyle 1m 50s trunk passed +1 mvnsite 2m 20s trunk passed +1 mvneclipse 0m 46s trunk passed +1 findbugs 3m 37s trunk passed +1 javadoc 1m 41s trunk passed 0 mvndep 0m 15s Maven dependency ordering for patch +1 mvninstall 1m 36s the patch passed +1 compile 10m 55s the patch passed +1 javac 10m 55s the patch passed +1 checkstyle 1m 42s the patch passed +1 mvnsite 2m 22s the patch passed +1 mvneclipse 0m 41s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 4m 6s the patch passed +1 javadoc 1m 43s the patch passed +1 unit 8m 36s hadoop-common in the patch passed. +1 unit 79m 20s hadoop-hdfs in the patch passed. +1 asflicense 0m 51s The patch does not generate ASF License warnings. 143m 16s Subsystem Report/Notes Docker Image:yetus/hadoop:e809691 JIRA Issue HDFS-11100 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12838384/HDFS-11100.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux df190ab3d1ce 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ca68f9c Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17507/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17507/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 16s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 1m 51s Maven dependency ordering for branch
        +1 mvninstall 8m 15s trunk passed
        -1 compile 8m 3s root in trunk failed.
        +1 checkstyle 1m 43s trunk passed
        +1 mvnsite 2m 17s trunk passed
        +1 mvneclipse 0m 46s trunk passed
        +1 findbugs 4m 1s trunk passed
        +1 javadoc 1m 44s trunk passed
        0 mvndep 0m 15s Maven dependency ordering for patch
        +1 mvninstall 1m 33s the patch passed
        -1 compile 6m 26s root in the patch failed.
        -1 javac 6m 26s root in the patch failed.
        +1 checkstyle 1m 35s the patch passed
        +1 mvnsite 2m 6s the patch passed
        +1 mvneclipse 0m 40s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 3m 50s the patch passed
        +1 javadoc 1m 41s the patch passed
        +1 unit 8m 33s hadoop-common in the patch passed.
        -1 unit 58m 40s hadoop-hdfs in the patch failed.
        +1 asflicense 1m 12s The patch does not generate ASF License warnings.
        116m 48s



        Reason Tests
        Failed junit tests hadoop.hdfs.tools.TestDFSAdmin



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:e809691
        JIRA Issue HDFS-11100
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12838407/HDFS-11100.002.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux fd5422ca09c8 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 89354f0
        Default Java 1.8.0_101
        compile https://builds.apache.org/job/PreCommit-HDFS-Build/17510/artifact/patchprocess/branch-compile-root.txt
        findbugs v3.0.0
        compile https://builds.apache.org/job/PreCommit-HDFS-Build/17510/artifact/patchprocess/patch-compile-root.txt
        javac https://builds.apache.org/job/PreCommit-HDFS-Build/17510/artifact/patchprocess/patch-compile-root.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/17510/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17510/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17510/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 1m 51s Maven dependency ordering for branch +1 mvninstall 8m 15s trunk passed -1 compile 8m 3s root in trunk failed. +1 checkstyle 1m 43s trunk passed +1 mvnsite 2m 17s trunk passed +1 mvneclipse 0m 46s trunk passed +1 findbugs 4m 1s trunk passed +1 javadoc 1m 44s trunk passed 0 mvndep 0m 15s Maven dependency ordering for patch +1 mvninstall 1m 33s the patch passed -1 compile 6m 26s root in the patch failed. -1 javac 6m 26s root in the patch failed. +1 checkstyle 1m 35s the patch passed +1 mvnsite 2m 6s the patch passed +1 mvneclipse 0m 40s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 50s the patch passed +1 javadoc 1m 41s the patch passed +1 unit 8m 33s hadoop-common in the patch passed. -1 unit 58m 40s hadoop-hdfs in the patch failed. +1 asflicense 1m 12s The patch does not generate ASF License warnings. 116m 48s Reason Tests Failed junit tests hadoop.hdfs.tools.TestDFSAdmin Subsystem Report/Notes Docker Image:yetus/hadoop:e809691 JIRA Issue HDFS-11100 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12838407/HDFS-11100.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux fd5422ca09c8 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 89354f0 Default Java 1.8.0_101 compile https://builds.apache.org/job/PreCommit-HDFS-Build/17510/artifact/patchprocess/branch-compile-root.txt findbugs v3.0.0 compile https://builds.apache.org/job/PreCommit-HDFS-Build/17510/artifact/patchprocess/patch-compile-root.txt javac https://builds.apache.org/job/PreCommit-HDFS-Build/17510/artifact/patchprocess/patch-compile-root.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/17510/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17510/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17510/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        jzhuge John Zhuge added a comment -

        Rekick the tests.

        Show
        jzhuge John Zhuge added a comment - Rekick the tests.
        Hide
        yzhangal Yongjun Zhang added a comment -

        Hi John Zhuge,

        Thanks for reporting the issue and the patch. I did a review and it looks good overall. I have the following comments:

        1. Some optimization can be done about the sticky bit checking, to avoid unnecessary memory allocations (assuming all children of the same dir contains same number of components in their paths, and share the same parent path components):

                INodeAttributes[] childInodeAttrs = null;
                byte[][] childComponents = null;
                int childCompIdx = 0;
                for(INode child : cList) {
                  if (childComponents == null) {
                    childComponents = child.getPathComponents();
                    childCompIdx = childComponents.length-1;
                  } else {
                    childComponents[childCompIdx] =
                        child.getLocalNameBytes();
                  }
                  if (childInodeAttrs == null) {
                    childInodeAttrs =
                        new INodeAttributes[childComponents.length];
                  }
        

        2. testStickyBitRecursiveDeleteFile() and testStickyBitRecursiveDeleteDir() have quite some code in common, suggest to consolidate. At least the bottom part starting from "try" can be shared. The only diff is whether the child is a file or directory. We may not even need to create file under "dir" for testStickyBitRecursiveDeleteDir().

        3. The tests are to make sure that exception is throw when sticky bit is set and deletion is run with different user. Can we add another positive test that when sticky bit is set and the deletion is run successfully when as the same user?

        4. I noticed that linux may continue to delete the children recursively even if a parent is protected by sticky bit. HDFS may have a different semantics here, that is, it will throw an exception without going down the tree. I just want to make a note here.

        Thanks.

        Show
        yzhangal Yongjun Zhang added a comment - Hi John Zhuge , Thanks for reporting the issue and the patch. I did a review and it looks good overall. I have the following comments: 1. Some optimization can be done about the sticky bit checking, to avoid unnecessary memory allocations (assuming all children of the same dir contains same number of components in their paths, and share the same parent path components): INodeAttributes[] childInodeAttrs = null ; byte [][] childComponents = null ; int childCompIdx = 0; for (INode child : cList) { if (childComponents == null ) { childComponents = child.getPathComponents(); childCompIdx = childComponents.length-1; } else { childComponents[childCompIdx] = child.getLocalNameBytes(); } if (childInodeAttrs == null ) { childInodeAttrs = new INodeAttributes[childComponents.length]; } 2. testStickyBitRecursiveDeleteFile() and testStickyBitRecursiveDeleteDir() have quite some code in common, suggest to consolidate. At least the bottom part starting from "try" can be shared. The only diff is whether the child is a file or directory. We may not even need to create file under "dir" for testStickyBitRecursiveDeleteDir() . 3. The tests are to make sure that exception is throw when sticky bit is set and deletion is run with different user. Can we add another positive test that when sticky bit is set and the deletion is run successfully when as the same user? 4. I noticed that linux may continue to delete the children recursively even if a parent is protected by sticky bit. HDFS may have a different semantics here, that is, it will throw an exception without going down the tree. I just want to make a note here. Thanks.
        Hide
        jzhuge John Zhuge added a comment -

        Yongjun Zhang Thanks for the great review! Item 4 may be hard implement from my quick check of the code.

        Show
        jzhuge John Zhuge added a comment - Yongjun Zhang Thanks for the great review! Item 4 may be hard implement from my quick check of the code.
        Hide
        yzhangal Yongjun Zhang added a comment -

        Sure John Zhuge, I did not mean that we need to implement item 4 here. Just wanted to make a note of it. Thanks.

        Show
        yzhangal Yongjun Zhang added a comment - Sure John Zhuge , I did not mean that we need to implement item 4 here. Just wanted to make a note of it. Thanks.
        Hide
        jzhuge John Zhuge added a comment -

        1. Fixed.
        2. Tried but didn't find it helpful.
        3. Positive tests already exists at the end of each test case.

        Show
        jzhuge John Zhuge added a comment - 1. Fixed. 2. Tried but didn't find it helpful. 3. Positive tests already exists at the end of each test case.
        Hide
        jzhuge John Zhuge added a comment -

        Patch 003:

        • Optimization suggested by Yongjun
        Show
        jzhuge John Zhuge added a comment - Patch 003: Optimization suggested by Yongjun
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 11s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 1m 54s Maven dependency ordering for branch
        +1 mvninstall 7m 36s trunk passed
        +1 compile 10m 12s trunk passed
        +1 checkstyle 1m 38s trunk passed
        +1 mvnsite 2m 15s trunk passed
        +1 mvneclipse 0m 37s trunk passed
        +1 findbugs 3m 17s trunk passed
        +1 javadoc 1m 36s trunk passed
        0 mvndep 0m 15s Maven dependency ordering for patch
        +1 mvninstall 1m 22s the patch passed
        +1 compile 9m 28s the patch passed
        +1 javac 9m 28s the patch passed
        +1 checkstyle 1m 33s the patch passed
        +1 mvnsite 1m 59s the patch passed
        +1 mvneclipse 0m 38s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 3m 49s the patch passed
        +1 javadoc 1m 34s the patch passed
        -1 unit 7m 27s hadoop-common in the patch failed.
        -1 unit 65m 43s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 33s The patch does not generate ASF License warnings.
        124m 50s



        Reason Tests
        Failed junit tests hadoop.security.token.delegation.web.TestWebDelegationToken
          hadoop.hdfs.TestTrashWithSecureEncryptionZones
          hadoop.hdfs.TestSecureEncryptionZoneWithKMS



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11100
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12842698/HDFS-11100.003.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 484957378e77 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 4c38f11
        Default Java 1.8.0_111
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/17824/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/17824/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17824/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17824/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 1m 54s Maven dependency ordering for branch +1 mvninstall 7m 36s trunk passed +1 compile 10m 12s trunk passed +1 checkstyle 1m 38s trunk passed +1 mvnsite 2m 15s trunk passed +1 mvneclipse 0m 37s trunk passed +1 findbugs 3m 17s trunk passed +1 javadoc 1m 36s trunk passed 0 mvndep 0m 15s Maven dependency ordering for patch +1 mvninstall 1m 22s the patch passed +1 compile 9m 28s the patch passed +1 javac 9m 28s the patch passed +1 checkstyle 1m 33s the patch passed +1 mvnsite 1m 59s the patch passed +1 mvneclipse 0m 38s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 49s the patch passed +1 javadoc 1m 34s the patch passed -1 unit 7m 27s hadoop-common in the patch failed. -1 unit 65m 43s hadoop-hdfs in the patch failed. +1 asflicense 0m 33s The patch does not generate ASF License warnings. 124m 50s Reason Tests Failed junit tests hadoop.security.token.delegation.web.TestWebDelegationToken   hadoop.hdfs.TestTrashWithSecureEncryptionZones   hadoop.hdfs.TestSecureEncryptionZoneWithKMS Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11100 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12842698/HDFS-11100.003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 484957378e77 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 4c38f11 Default Java 1.8.0_111 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/17824/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/17824/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17824/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17824/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        jzhuge John Zhuge added a comment -

        Test failures are unrelated and tracked by HADOOP-13890.

        Show
        jzhuge John Zhuge added a comment - Test failures are unrelated and tracked by HADOOP-13890 .
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Note that HDFS is not meant to be Linux compatible, rather, it's meant to be POSIX-compatible

        Quoting Hadoop Security on sticky bit:

        ... it [sticky bit] means that files in a directory can only be deleted by the owner of that file. Without the sticky bit set, a file can be deleted by anyone that has write access to the directory. In HDFS, the owner of a directory and the HDFS superuser can also delete files regardless of whether the sticky bit is set. The sticky bit is useful for directories, such as /tmp, where you want all users to have write access to the directory but only the owner of the data should be able to delete data.

        The Linux's behavior is documented/defined here: http://man7.org/linux/man-pages/man1/chmod.1.html

        RESTRICTED DELETION FLAG OR STICKY BIT

        The restricted deletion flag or sticky bit is a single bit, whose
        interpretation depends on the file type. For directories, it
        prevents unprivileged users from removing or renaming a file in the
        directory unless they own the file or the directory; this is called
        the restricted deletion flag for the directory, and is commonly found
        on world-writable directories like /tmp. For regular files on some
        older systems, the bit saves the program's text image on the swap
        device so it will load more quickly when run; this is called the
        sticky bit.

        However, I found no mention of sticky bit handling in Open Group specification regarding rm command: http://pubs.opengroup.org/onlinepubs/009695399/utilities/rm.html

        This wikipage describes the behavior of removing files under sticky-bit attributed directories: https://en.wikipedia.org/wiki/Sticky_bit
        As you can see, there is no common behavior across all Unix-like operating systems. But for operating systems that use sticky bit to protect files removals, HDFS's behavior is definitely different. So I think it's safe to say the old behavior is unexpected to most Unix-like users, and the new behavior is what most Unix-like users would expect.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Note that HDFS is not meant to be Linux compatible , rather, it's meant to be POSIX-compatible Quoting Hadoop Security on sticky bit: ... it [sticky bit] means that files in a directory can only be deleted by the owner of that file. Without the sticky bit set, a file can be deleted by anyone that has write access to the directory. In HDFS, the owner of a directory and the HDFS superuser can also delete files regardless of whether the sticky bit is set. The sticky bit is useful for directories, such as /tmp, where you want all users to have write access to the directory but only the owner of the data should be able to delete data. The Linux's behavior is documented/defined here: http://man7.org/linux/man-pages/man1/chmod.1.html RESTRICTED DELETION FLAG OR STICKY BIT The restricted deletion flag or sticky bit is a single bit, whose interpretation depends on the file type. For directories, it prevents unprivileged users from removing or renaming a file in the directory unless they own the file or the directory; this is called the restricted deletion flag for the directory, and is commonly found on world-writable directories like /tmp. For regular files on some older systems, the bit saves the program's text image on the swap device so it will load more quickly when run; this is called the sticky bit. However, I found no mention of sticky bit handling in Open Group specification regarding rm command: http://pubs.opengroup.org/onlinepubs/009695399/utilities/rm.html This wikipage describes the behavior of removing files under sticky-bit attributed directories: https://en.wikipedia.org/wiki/Sticky_bit As you can see, there is no common behavior across all Unix-like operating systems. But for operating systems that use sticky bit to protect files removals, HDFS's behavior is definitely different. So I think it's safe to say the old behavior is unexpected to most Unix-like users, and the new behavior is what most Unix-like users would expect.
        Hide
        jojochuang Wei-Chiu Chuang added a comment - - edited

        Thanks John Zhuge for the code contribution! I've completed my first review of the patch.

        My review notes are as follows:

        1. The patch adds an extra for loop and we should be careful to avoid pausing NameNode from other operations for too long.
        2. Given that the original method body is essentially a breadth-first-search (edit: my bad. This is a DFS), it should be easy to separate from the new code, could we refactor the new code into a new method, to make it easier to understand?
        3. One nit: the following line is repeated for every child inodes which only need to be done once. Performance-wise this shouldn't have much impact.
                      // checkStickyBit only uses 2 entries in childInodeAttrs
                      childInodeAttrs[parentIdx] = inodeAttr;
          
        4. In the patch, an INodeAttributes array is instantiated per INodeDirectory. This can be a performance hitter. Would it make sense to optimize it? For example, pre-initialize this array, and double its length if childComponents.length is larger than childInodeAttrs.length. We can file a new jira for the optimization if you think this is reasonable.
        Show
        jojochuang Wei-Chiu Chuang added a comment - - edited Thanks John Zhuge for the code contribution! I've completed my first review of the patch. My review notes are as follows: The patch adds an extra for loop and we should be careful to avoid pausing NameNode from other operations for too long. Given that the original method body is essentially a breadth-first-search (edit: my bad. This is a DFS), it should be easy to separate from the new code, could we refactor the new code into a new method, to make it easier to understand? One nit: the following line is repeated for every child inodes which only need to be done once. Performance-wise this shouldn't have much impact. // checkStickyBit only uses 2 entries in childInodeAttrs childInodeAttrs[parentIdx] = inodeAttr; In the patch, an INodeAttributes array is instantiated per INodeDirectory. This can be a performance hitter. Would it make sense to optimize it? For example, pre-initialize this array, and double its length if childComponents.length is larger than childInodeAttrs.length . We can file a new jira for the optimization if you think this is reasonable.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Let's push this back to the next 3.0 release. Don't think it'll get in in time.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Let's push this back to the next 3.0 release. Don't think it'll get in in time.
        Hide
        jzhuge John Zhuge added a comment -

        Patch 004

        • Performance optimization by only performing expensive operations right before throwing an exception
        Show
        jzhuge John Zhuge added a comment - Patch 004 Performance optimization by only performing expensive operations right before throwing an exception
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 17s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 17s Maven dependency ordering for branch
        +1 mvninstall 14m 10s trunk passed
        +1 compile 13m 45s trunk passed
        +1 checkstyle 1m 34s trunk passed
        +1 mvnsite 1m 59s trunk passed
        +1 mvneclipse 0m 37s trunk passed
        +1 findbugs 3m 24s trunk passed
        +1 javadoc 1m 34s trunk passed
        0 mvndep 0m 14s Maven dependency ordering for patch
        +1 mvninstall 1m 33s the patch passed
        +1 compile 11m 14s the patch passed
        +1 javac 11m 14s the patch passed
        -0 checkstyle 1m 31s root: The patch generated 1 new + 50 unchanged - 0 fixed = 51 total (was 50)
        +1 mvnsite 1m 57s the patch passed
        +1 mvneclipse 0m 37s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 3m 35s the patch passed
        +1 javadoc 1m 30s the patch passed
        +1 unit 7m 50s hadoop-common in the patch passed.
        -1 unit 88m 45s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 58s The patch does not generate ASF License warnings.
        158m 34s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
          hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
        Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11100
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12851323/HDFS-11100.004.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 5859fe88ac1f 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / d88497d
        Default Java 1.8.0_121
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18335/artifact/patchprocess/diff-checkstyle-root.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/18335/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18335/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18335/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 17s Maven dependency ordering for branch +1 mvninstall 14m 10s trunk passed +1 compile 13m 45s trunk passed +1 checkstyle 1m 34s trunk passed +1 mvnsite 1m 59s trunk passed +1 mvneclipse 0m 37s trunk passed +1 findbugs 3m 24s trunk passed +1 javadoc 1m 34s trunk passed 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 1m 33s the patch passed +1 compile 11m 14s the patch passed +1 javac 11m 14s the patch passed -0 checkstyle 1m 31s root: The patch generated 1 new + 50 unchanged - 0 fixed = 51 total (was 50) +1 mvnsite 1m 57s the patch passed +1 mvneclipse 0m 37s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 35s the patch passed +1 javadoc 1m 30s the patch passed +1 unit 7m 50s hadoop-common in the patch passed. -1 unit 88m 45s hadoop-hdfs in the patch failed. +1 asflicense 0m 58s The patch does not generate ASF License warnings. 158m 34s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2 Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11100 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12851323/HDFS-11100.004.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 5859fe88ac1f 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / d88497d Default Java 1.8.0_121 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18335/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/18335/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18335/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18335/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        jzhuge John Zhuge added a comment -

        Patch 005

        • Fix checkstyle
        Show
        jzhuge John Zhuge added a comment - Patch 005 Fix checkstyle
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 21s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 16s Maven dependency ordering for branch
        +1 mvninstall 12m 46s trunk passed
        +1 compile 12m 32s trunk passed
        +1 checkstyle 1m 34s trunk passed
        +1 mvnsite 2m 5s trunk passed
        +1 mvneclipse 0m 36s trunk passed
        +1 findbugs 3m 22s trunk passed
        +1 javadoc 1m 33s trunk passed
        0 mvndep 0m 14s Maven dependency ordering for patch
        +1 mvninstall 1m 30s the patch passed
        +1 compile 11m 56s the patch passed
        +1 javac 11m 56s the patch passed
        +1 checkstyle 1m 43s the patch passed
        +1 mvnsite 2m 11s the patch passed
        +1 mvneclipse 0m 41s the patch passed
        +1 whitespace 0m 1s The patch has no whitespace issues.
        +1 findbugs 4m 9s the patch passed
        +1 javadoc 1m 44s the patch passed
        -1 unit 9m 13s hadoop-common in the patch failed.
        -1 unit 73m 27s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 35s The patch does not generate ASF License warnings.
        143m 45s



        Reason Tests
        Failed junit tests hadoop.security.TestKDiag
          hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation
          hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
          hadoop.hdfs.server.datanode.TestDataNodeUUID



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11100
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12851411/HDFS-11100.005.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 9f561534a431 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 9dbfab1
        Default Java 1.8.0_121
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/18338/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/18338/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18338/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18338/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 21s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 16s Maven dependency ordering for branch +1 mvninstall 12m 46s trunk passed +1 compile 12m 32s trunk passed +1 checkstyle 1m 34s trunk passed +1 mvnsite 2m 5s trunk passed +1 mvneclipse 0m 36s trunk passed +1 findbugs 3m 22s trunk passed +1 javadoc 1m 33s trunk passed 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 1m 30s the patch passed +1 compile 11m 56s the patch passed +1 javac 11m 56s the patch passed +1 checkstyle 1m 43s the patch passed +1 mvnsite 2m 11s the patch passed +1 mvneclipse 0m 41s the patch passed +1 whitespace 0m 1s The patch has no whitespace issues. +1 findbugs 4m 9s the patch passed +1 javadoc 1m 44s the patch passed -1 unit 9m 13s hadoop-common in the patch failed. -1 unit 73m 27s hadoop-hdfs in the patch failed. +1 asflicense 0m 35s The patch does not generate ASF License warnings. 143m 45s Reason Tests Failed junit tests hadoop.security.TestKDiag   hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.server.datanode.TestDataNodeUUID Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11100 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12851411/HDFS-11100.005.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 9f561534a431 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 9dbfab1 Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/18338/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/18338/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18338/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18338/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        +1. The failed test are unrelated and can't be reproduced locally.
        Thanks John Zhuge for the nice work!

        Show
        jojochuang Wei-Chiu Chuang added a comment - +1. The failed test are unrelated and can't be reproduced locally. Thanks John Zhuge for the nice work!
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        I will commit the 005 patch to trunk by end of day if there are no objections.

        Show
        jojochuang Wei-Chiu Chuang added a comment - I will commit the 005 patch to trunk by end of day if there are no objections.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Committed 005 patch to trunk.
        Much thanks to John Zhuge for the patch and Yongjun Zhang for the initial review!

        Show
        jojochuang Wei-Chiu Chuang added a comment - Committed 005 patch to trunk. Much thanks to John Zhuge for the patch and Yongjun Zhang for the initial review!
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11265 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11265/)
        HDFS-11100. Recursively deleting file protected by sticky bit should (weichiu: rev 5690b51ef7c708c0a71162ddaff04466bc71cdcc)

        • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSExceptionMessages.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/permission/TestStickyBit.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSPermissionChecker.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11265 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11265/ ) HDFS-11100 . Recursively deleting file protected by sticky bit should (weichiu: rev 5690b51ef7c708c0a71162ddaff04466bc71cdcc) (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSExceptionMessages.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/permission/TestStickyBit.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSPermissionChecker.java
        Hide
        jzhuge John Zhuge added a comment -

        Thanks Wei-Chiu Chuang and Yongjun Zhang for the review and commit!

        Show
        jzhuge John Zhuge added a comment - Thanks Wei-Chiu Chuang and Yongjun Zhang for the review and commit!
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        John Zhuge Since this is an incompatible change, if you can, please review/update my release note.

        Show
        jojochuang Wei-Chiu Chuang added a comment - John Zhuge Since this is an incompatible change, if you can, please review/update my release note.

          People

          • Assignee:
            jzhuge John Zhuge
            Reporter:
            jzhuge John Zhuge
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development