Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10689

Hdfs dfs chmod should reset sticky bit permission when the bit is omitted in the octal mode

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.6.4
    • Fix Version/s: 3.0.0-alpha1
    • Component/s: fs
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Incompatible change
    • Release Note:
      Hide
      Hdfs dfs chmod command will reset sticky bit permission on a file/directory when the leading sticky bit is omitted in the octal mode (like 644). So when a file/directory permission is applied using octal mode and sticky bit permission needs to be preserved, then it has to be explicitly mentioned in the permission bits (like 1644). This behavior is similar to many other filesystems on Linux/BSD.
      Show
      Hdfs dfs chmod command will reset sticky bit permission on a file/directory when the leading sticky bit is omitted in the octal mode (like 644). So when a file/directory permission is applied using octal mode and sticky bit permission needs to be preserved, then it has to be explicitly mentioned in the permission bits (like 1644). This behavior is similar to many other filesystems on Linux/BSD.

      Description

      When a directory permission is modified using hdfs dfs chmod command and when octal/numeric format is used, the leading sticky bit is not fully honored.

      1. Create a dir dir_test_with_sticky_bit
      2. Apply sticky bit permission on the dir : hdfs dfs -chmod 1755 /dir_test_with_sticky_bit
      3. Remove sticky bit permission on the dir: hdfs dfs -chmod 755 /dir_test_with_sticky_bit

      Expected: Remove the sticky bit on the dir, as it happens on Mac/Linux native filesystem with native chmod.

      4. However, removing sticky bit permission by explicitly turning off the bit works. hdfs dfs -chmod 0755 /dir_test_with_sticky_bit

      manoj@~/work/hadev-pp: hdfs dfs -chmod 1755 /dir_test_with_sticky_bit
      manoj@~/work/hadev-pp: hdfs dfs -ls /
      Found 2 items
      drwxr-xr-t   - manoj supergroup          0 2016-07-25 11:42 /dir_test_with_sticky_bit
      drwxr-xr-x   - manoj supergroup          0 2016-07-25 11:42 /user
      
      manoj@~/work/hadev-pp: hdfs dfs -chmod 755 /dir_test_with_sticky_bit
      manoj@~/work/hadev-pp: hdfs dfs -ls /
      Found 2 items
      drwxr-xr-t   - manoj supergroup          0 2016-07-25 11:42 /dir_test_with_sticky_bit  <=== sticky bit still intact
      drwxr-xr-x   - manoj supergroup          0 2016-07-25 11:42 /user
      
      manoj@~/work/hadev-pp: hdfs dfs -chmod 0755 /dir_test_with_sticky_bit
      manoj@~/work/hadev-pp: hdfs dfs -ls /
      Found 2 items
      drwxr-xr-x   - manoj supergroup          0 2016-07-25 11:42 /dir_test_with_sticky_bit
      drwxr-xr-x   - manoj supergroup          0 2016-07-25 11:42 /user
      manoj@~/work/hadev-pp: 
      
      1. HDFS-10689.001.patch
        11 kB
        Manoj Govindassamy
      2. HDFS-10689.002.patch
        9 kB
        Manoj Govindassamy
      3. HDFS-10689.003.patch
        9 kB
        Manoj Govindassamy
      4. HDFS-10689.004.patch
        9 kB
        Manoj Govindassamy

        Activity

        Hide
        manojg Manoj Govindassamy added a comment -

        Whenever the leading bit in the numeric format is omitted, it is considered to be all 0s. Chmod man page clearly mentions on the expected behavior with permission apply via numeric option with respect to setuid and setgid bits.

        man chmod:

        SETUID AND SETGID BITS
               chmod clears the set-group-ID bit of a regular file if the file’s group ID does not match the user’s effective group ID or one of  the  user’s  supplementary  group
               IDs,  unless  the  user  has  appropriate privileges.  Additional restrictions may cause the set-user-ID and set-group-ID bits of MODE or RFILE to be ignored.  This
               behavior depends on the policy and functionality of the underlying chmod system call.  When in doubt, check the underlying system behavior.
        
               chmod preserves a directory’s set-user-ID and set-group-ID bits unless you explicitly specify otherwise.  *You can set or clear the bits with symbolic modes like u+s
               and g-s, and you can set (but not clear) the bits with a numeric mode.*
        

        That is, 755 will not reset setuid and setgid bits on the file/directory. However, man page doesn't mention anything of that sort for sticky bits. So, I am leaning towards how other distros have implemented this. Here is the EXT4 behavior.

        -bash-4.1$ df -T
        Filesystem     Type  1K-blocks      Used Available Use% Mounted on
        /dev/xvda1     ext4  103209948  91990624   5977880  94% /
        
        -bash-4.1$ pwd
        /home/manojg
        
        drwxrwxr-x 2 manojg manojg 4096 Jul 25 19:09 dir_test_sticky_bit
        -bash-4.1$ chmod 1775 dir_test_sticky_bit
        -bash-4.1$ ls -l
        total 4
        drwxrwxr-t 2 manojg manojg 4096 Jul 25 19:09 dir_test_sticky_bit
        
        -bash-4.1$ chmod 775 dir_test_sticky_bit
        -bash-4.1$ ls -l
        total 4
        drwxrwxr-x 2 manojg manojg 4096 Jul 25 19:09 dir_test_sticky_bit  <=== 755 does clear out sticky bit
        -bash-4.1$ .
        

        So, EXT4 and many other filesystems on Linux, MacOSX are resetting Sticky Bit when the bit is not specified in the permission arg.

        Show
        manojg Manoj Govindassamy added a comment - Whenever the leading bit in the numeric format is omitted, it is considered to be all 0s. Chmod man page clearly mentions on the expected behavior with permission apply via numeric option with respect to setuid and setgid bits. man chmod: SETUID AND SETGID BITS chmod clears the set-group-ID bit of a regular file if the file’s group ID does not match the user’s effective group ID or one of the user’s supplementary group IDs, unless the user has appropriate privileges. Additional restrictions may cause the set-user-ID and set-group-ID bits of MODE or RFILE to be ignored. This behavior depends on the policy and functionality of the underlying chmod system call. When in doubt, check the underlying system behavior. chmod preserves a directory’s set-user-ID and set-group-ID bits unless you explicitly specify otherwise. *You can set or clear the bits with symbolic modes like u+s and g-s, and you can set (but not clear) the bits with a numeric mode.* That is, 755 will not reset setuid and setgid bits on the file/directory. However, man page doesn't mention anything of that sort for sticky bits. So, I am leaning towards how other distros have implemented this. Here is the EXT4 behavior. -bash-4.1$ df -T Filesystem Type 1K-blocks Used Available Use% Mounted on /dev/xvda1 ext4 103209948 91990624 5977880 94% / -bash-4.1$ pwd /home/manojg drwxrwxr-x 2 manojg manojg 4096 Jul 25 19:09 dir_test_sticky_bit -bash-4.1$ chmod 1775 dir_test_sticky_bit -bash-4.1$ ls -l total 4 drwxrwxr-t 2 manojg manojg 4096 Jul 25 19:09 dir_test_sticky_bit -bash-4.1$ chmod 775 dir_test_sticky_bit -bash-4.1$ ls -l total 4 drwxrwxr-x 2 manojg manojg 4096 Jul 25 19:09 dir_test_sticky_bit <=== 755 does clear out sticky bit -bash-4.1$ . So, EXT4 and many other filesystems on Linux, MacOSX are resetting Sticky Bit when the bit is not specified in the permission arg.
        Hide
        andrew.wang Andrew Wang added a comment -

        Thanks for digging into this Manoj. I'm of the opinion that we should try to imitate the Linux/BSD behavior, since that'll be least surprising to end users.

        Now the question is if we declare this a bug fix that can be backported to branch-2, or if this behavior change is too incompatible. Given that sticky bits are pretty rare in general, I think it's safe for branch-2, but would welcome other's thoughts. Anything to add Chris Nauroth?

        Show
        andrew.wang Andrew Wang added a comment - Thanks for digging into this Manoj. I'm of the opinion that we should try to imitate the Linux/BSD behavior, since that'll be least surprising to end users. Now the question is if we declare this a bug fix that can be backported to branch-2, or if this behavior change is too incompatible. Given that sticky bits are pretty rare in general, I think it's safe for branch-2, but would welcome other's thoughts. Anything to add Chris Nauroth ?
        Hide
        manojg Manoj Govindassamy added a comment -

        Attaching a patch to make hdfs dfs chmod handle sticky bit permissions just like other filesystems.

        Test results with the proposed fix:

        manoj@~/work/hadev-pp: hdfs dfs -ls /
        Found 1 items
        drwxr-xr-x   - manoj supergroup          0 2016-07-26 15:13 /user
        manoj@~/work/hadev-pp: hdfs dfs -mkdir /test_dir
        manoj@~/work/hadev-pp: hdfs dfs -ls /
        Found 2 items
        drwxr-xr-x   - manoj supergroup          0 2016-07-26 15:15 /test_dir
        drwxr-xr-x   - manoj supergroup          0 2016-07-26 15:13 /user
        manoj@~/work/hadev-pp: hdfs dfs -chmod +t /test_dir
        manoj@~/work/hadev-pp: hdfs dfs -ls /
        Found 2 items
        drwxr-xr-t   - manoj supergroup          0 2016-07-26 15:15 /test_dir
        drwxr-xr-x   - manoj supergroup          0 2016-07-26 15:13 /user
        manoj@~/work/hadev-pp: hdfs dfs -chmod -t /test_dir
        manoj@~/work/hadev-pp: hdfs dfs -ls /
        Found 2 items
        drwxr-xr-x   - manoj supergroup          0 2016-07-26 15:15 /test_dir
        drwxr-xr-x   - manoj supergroup          0 2016-07-26 15:13 /user
        manoj@~/work/hadev-pp: hdfs dfs -chmod 755 /test_dir
        manoj@~/work/hadev-pp: hdfs dfs -ls /
        Found 2 items
        drwxr-xr-x   - manoj supergroup          0 2016-07-26 15:15 /test_dir
        drwxr-xr-x   - manoj supergroup          0 2016-07-26 15:13 /user
        manoj@~/work/hadev-pp: hdfs dfs -chmod 1755 /test_dir
        manoj@~/work/hadev-pp: hdfs dfs -ls /
        Found 2 items
        drwxr-xr-t   - manoj supergroup          0 2016-07-26 15:15 /test_dir
        drwxr-xr-x   - manoj supergroup          0 2016-07-26 15:13 /user
        manoj@~/work/hadev-pp: hdfs dfs -chmod 0755 /test_dir
        manoj@~/work/hadev-pp: hdfs dfs -ls /
        Found 2 items
        drwxr-xr-x   - manoj supergroup          0 2016-07-26 15:15 /test_dir
        drwxr-xr-x   - manoj supergroup          0 2016-07-26 15:13 /user
        manoj@~/work/hadev-pp: hdfs dfs -chmod 1755 /test_dir
        manoj@~/work/hadev-pp: hdfs dfs -ls /
        Found 2 items
        drwxr-xr-t   - manoj supergroup          0 2016-07-26 15:15 /test_dir
        drwxr-xr-x   - manoj supergroup          0 2016-07-26 15:13 /user
        manoj@~/work/hadev-pp: hdfs dfs -chmod 755 /test_dir
        manoj@~/work/hadev-pp: hdfs dfs -ls /
        Found 2 items
        drwxr-xr-x   - manoj supergroup          0 2016-07-26 15:15 /test_dir
        drwxr-xr-x   - manoj supergroup          0 2016-07-26 15:13 /user
        
        Show
        manojg Manoj Govindassamy added a comment - Attaching a patch to make hdfs dfs chmod handle sticky bit permissions just like other filesystems. Test results with the proposed fix: manoj@~/work/hadev-pp: hdfs dfs -ls / Found 1 items drwxr-xr-x - manoj supergroup 0 2016-07-26 15:13 /user manoj@~/work/hadev-pp: hdfs dfs -mkdir /test_dir manoj@~/work/hadev-pp: hdfs dfs -ls / Found 2 items drwxr-xr-x - manoj supergroup 0 2016-07-26 15:15 /test_dir drwxr-xr-x - manoj supergroup 0 2016-07-26 15:13 /user manoj@~/work/hadev-pp: hdfs dfs -chmod +t /test_dir manoj@~/work/hadev-pp: hdfs dfs -ls / Found 2 items drwxr-xr-t - manoj supergroup 0 2016-07-26 15:15 /test_dir drwxr-xr-x - manoj supergroup 0 2016-07-26 15:13 /user manoj@~/work/hadev-pp: hdfs dfs -chmod -t /test_dir manoj@~/work/hadev-pp: hdfs dfs -ls / Found 2 items drwxr-xr-x - manoj supergroup 0 2016-07-26 15:15 /test_dir drwxr-xr-x - manoj supergroup 0 2016-07-26 15:13 /user manoj@~/work/hadev-pp: hdfs dfs -chmod 755 /test_dir manoj@~/work/hadev-pp: hdfs dfs -ls / Found 2 items drwxr-xr-x - manoj supergroup 0 2016-07-26 15:15 /test_dir drwxr-xr-x - manoj supergroup 0 2016-07-26 15:13 /user manoj@~/work/hadev-pp: hdfs dfs -chmod 1755 /test_dir manoj@~/work/hadev-pp: hdfs dfs -ls / Found 2 items drwxr-xr-t - manoj supergroup 0 2016-07-26 15:15 /test_dir drwxr-xr-x - manoj supergroup 0 2016-07-26 15:13 /user manoj@~/work/hadev-pp: hdfs dfs -chmod 0755 /test_dir manoj@~/work/hadev-pp: hdfs dfs -ls / Found 2 items drwxr-xr-x - manoj supergroup 0 2016-07-26 15:15 /test_dir drwxr-xr-x - manoj supergroup 0 2016-07-26 15:13 /user manoj@~/work/hadev-pp: hdfs dfs -chmod 1755 /test_dir manoj@~/work/hadev-pp: hdfs dfs -ls / Found 2 items drwxr-xr-t - manoj supergroup 0 2016-07-26 15:15 /test_dir drwxr-xr-x - manoj supergroup 0 2016-07-26 15:13 /user manoj@~/work/hadev-pp: hdfs dfs -chmod 755 /test_dir manoj@~/work/hadev-pp: hdfs dfs -ls / Found 2 items drwxr-xr-x - manoj supergroup 0 2016-07-26 15:15 /test_dir drwxr-xr-x - manoj supergroup 0 2016-07-26 15:13 /user
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 20s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        0 mvndep 1m 38s Maven dependency ordering for branch
        +1 mvninstall 6m 49s trunk passed
        +1 compile 6m 47s trunk passed
        +1 checkstyle 1m 27s trunk passed
        +1 mvnsite 1m 50s trunk passed
        +1 mvneclipse 0m 26s trunk passed
        +1 findbugs 3m 9s trunk passed
        +1 javadoc 1m 41s trunk passed
        0 mvndep 0m 14s Maven dependency ordering for patch
        +1 mvninstall 1m 29s the patch passed
        +1 compile 7m 4s the patch passed
        +1 javac 7m 4s the patch passed
        -0 checkstyle 1m 33s root: The patch generated 11 new + 268 unchanged - 2 fixed = 279 total (was 270)
        +1 mvnsite 1m 57s the patch passed
        +1 mvneclipse 0m 27s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 3m 39s the patch passed
        +1 javadoc 1m 40s the patch passed
        +1 unit 7m 8s hadoop-common in the patch passed.
        -1 unit 58m 7s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 23s The patch does not generate ASF License warnings.
        108m 39s



        Reason Tests
        Failed junit tests hadoop.cli.TestHDFSCLI



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820332/HDFS-10689.001.patch
        JIRA Issue HDFS-10689
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 8fc8ab6c6d25 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / d84ab8a
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16202/artifact/patchprocess/diff-checkstyle-root.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16202/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16202/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16202/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 1m 38s Maven dependency ordering for branch +1 mvninstall 6m 49s trunk passed +1 compile 6m 47s trunk passed +1 checkstyle 1m 27s trunk passed +1 mvnsite 1m 50s trunk passed +1 mvneclipse 0m 26s trunk passed +1 findbugs 3m 9s trunk passed +1 javadoc 1m 41s trunk passed 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 1m 29s the patch passed +1 compile 7m 4s the patch passed +1 javac 7m 4s the patch passed -0 checkstyle 1m 33s root: The patch generated 11 new + 268 unchanged - 2 fixed = 279 total (was 270) +1 mvnsite 1m 57s the patch passed +1 mvneclipse 0m 27s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 39s the patch passed +1 javadoc 1m 40s the patch passed +1 unit 7m 8s hadoop-common in the patch passed. -1 unit 58m 7s hadoop-hdfs in the patch failed. +1 asflicense 0m 23s The patch does not generate ASF License warnings. 108m 39s Reason Tests Failed junit tests hadoop.cli.TestHDFSCLI Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820332/HDFS-10689.001.patch JIRA Issue HDFS-10689 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 8fc8ab6c6d25 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / d84ab8a Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16202/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16202/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16202/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16202/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        manojg Manoj Govindassamy added a comment -

        TestHDFSCLI failure is not related to the patch. Latest trunk is seeing the same failure.

        17699 2016-07-27 10:30:42,785 [main] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(186)) -
        17700 2016-07-27 10:30:42,785 [main] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(190)) - Summary results:
        17701 2016-07-27 10:30:42,785 [main] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(191)) - ----------------------------------
        17702 
        17703 2016-07-27 10:30:42,785 [main] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(211)) -                Testing mode: test
        17704 2016-07-27 10:30:42,785 [main] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(212)) -
        17705 2016-07-27 10:30:42,786 [main] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(213)) -              Overall result: --- FAIL ---
        17706 2016-07-27 10:30:42,786 [main] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(221)) -                # Tests pass: 665 (99%)
        17707 2016-07-27 10:30:42,786 [main] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(223)) -                # Tests fail: 1 (0%)
        17708 2016-07-27 10:30:42,786 [main] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(227)) -          # Validations done: 1726 (each test may do mu      ltiple validations)
        17709 2016-07-27 10:30:42,786 [main] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(230)) -
        17710 2016-07-27 10:30:42,786 [main] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(231)) - Failing tests:
        17711 2016-07-27 10:30:42,786 [main] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(232)) - --------------17712 2016-07-27 10:30:42,786 [main] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(238)) - 595: help: help for dfsadmin clrSpaceQuota
        17713 2016-07-27 10:30:42,787 [main] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(248)) -17714 2016-07-27 10:30:42,787 [main] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(249)) - Passing tests:
        17715 2016-07-27 10:30:42,787 [main] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(250)) - --------------
        17716 2016-07-27 10:30:42,787 [main] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(254)) - 1: ls: file using absolute path
        17717 2016-07-27 10:30:42,787 [main] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(254)) - 2: ls: file using relative path
        
        
        
        17646 2016-07-27 10:45:22,083 [main] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(177)) -                  Comparator: [RegexpComparator]
        17647 2016-07-27 10:45:22,083 [main] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(179)) -          Comparision result:   [fail]
        17648 2016-07-27 10:45:22,083 [main] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(181)) -             Expected output:   [^( |\t)*The storage ty      pe specific quota is cleared when -storageType option is specified.( )*]
        17649 2016-07-27 10:45:22,083 [main] INFO  cli.CLITestHelper (CLITestHelper.java:displayResults(183)) -               Actual output:   [-clrSpaceQuota [-stora      geType <storagetype>] <dirname>...<dirname>: Clear the space quota for each directory <dirName>.
        17650                 For each directory, attempt to clear the quota. An error will be reported if
        17651                 1. the directory does not exist or is a file, or
        17652                 2. user is not an administrator.
        17653                 It does not fault if the directory has no quota.
        17654                 The storage type specific quota is cleared when -storageType option is specified.               Available storageTypes are
        17655                 - RAM_DISK
        17656                 - DISK
        17657                 - SSD
        17658                 - ARCHIVE
        
        
        Show
        manojg Manoj Govindassamy added a comment - TestHDFSCLI failure is not related to the patch. Latest trunk is seeing the same failure. 17699 2016-07-27 10:30:42,785 [main] INFO cli.CLITestHelper (CLITestHelper.java:displayResults(186)) - 17700 2016-07-27 10:30:42,785 [main] INFO cli.CLITestHelper (CLITestHelper.java:displayResults(190)) - Summary results: 17701 2016-07-27 10:30:42,785 [main] INFO cli.CLITestHelper (CLITestHelper.java:displayResults(191)) - ---------------------------------- 17702 17703 2016-07-27 10:30:42,785 [main] INFO cli.CLITestHelper (CLITestHelper.java:displayResults(211)) - Testing mode: test 17704 2016-07-27 10:30:42,785 [main] INFO cli.CLITestHelper (CLITestHelper.java:displayResults(212)) - 17705 2016-07-27 10:30:42,786 [main] INFO cli.CLITestHelper (CLITestHelper.java:displayResults(213)) - Overall result: --- FAIL --- 17706 2016-07-27 10:30:42,786 [main] INFO cli.CLITestHelper (CLITestHelper.java:displayResults(221)) - # Tests pass: 665 (99%) 17707 2016-07-27 10:30:42,786 [main] INFO cli.CLITestHelper (CLITestHelper.java:displayResults(223)) - # Tests fail: 1 (0%) 17708 2016-07-27 10:30:42,786 [main] INFO cli.CLITestHelper (CLITestHelper.java:displayResults(227)) - # Validations done: 1726 (each test may do mu ltiple validations) 17709 2016-07-27 10:30:42,786 [main] INFO cli.CLITestHelper (CLITestHelper.java:displayResults(230)) - 17710 2016-07-27 10:30:42,786 [main] INFO cli.CLITestHelper (CLITestHelper.java:displayResults(231)) - Failing tests: 17711 2016-07-27 10:30:42,786 [main] INFO cli.CLITestHelper (CLITestHelper.java:displayResults(232)) - --------------17712 2016-07-27 10:30:42,786 [main] INFO cli.CLITestHelper (CLITestHelper.java:displayResults(238)) - 595: help: help for dfsadmin clrSpaceQuota 17713 2016-07-27 10:30:42,787 [main] INFO cli.CLITestHelper (CLITestHelper.java:displayResults(248)) -17714 2016-07-27 10:30:42,787 [main] INFO cli.CLITestHelper (CLITestHelper.java:displayResults(249)) - Passing tests: 17715 2016-07-27 10:30:42,787 [main] INFO cli.CLITestHelper (CLITestHelper.java:displayResults(250)) - -------------- 17716 2016-07-27 10:30:42,787 [main] INFO cli.CLITestHelper (CLITestHelper.java:displayResults(254)) - 1: ls: file using absolute path 17717 2016-07-27 10:30:42,787 [main] INFO cli.CLITestHelper (CLITestHelper.java:displayResults(254)) - 2: ls: file using relative path 17646 2016-07-27 10:45:22,083 [main] INFO cli.CLITestHelper (CLITestHelper.java:displayResults(177)) - Comparator: [RegexpComparator] 17647 2016-07-27 10:45:22,083 [main] INFO cli.CLITestHelper (CLITestHelper.java:displayResults(179)) - Comparision result: [fail] 17648 2016-07-27 10:45:22,083 [main] INFO cli.CLITestHelper (CLITestHelper.java:displayResults(181)) - Expected output: [^( |\t)*The storage ty pe specific quota is cleared when -storageType option is specified.( )*] 17649 2016-07-27 10:45:22,083 [main] INFO cli.CLITestHelper (CLITestHelper.java:displayResults(183)) - Actual output: [-clrSpaceQuota [-stora geType <storagetype>] <dirname>...<dirname>: Clear the space quota for each directory <dirName>. 17650 For each directory, attempt to clear the quota. An error will be reported if 17651 1. the directory does not exist or is a file, or 17652 2. user is not an administrator. 17653 It does not fault if the directory has no quota. 17654 The storage type specific quota is cleared when -storageType option is specified. Available storageTypes are 17655 - RAM_DISK 17656 - DISK 17657 - SSD 17658 - ARCHIVE
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        The test failure was filed in HDFS-10696, and it was caused by HDFS-9937.

        Show
        jojochuang Wei-Chiu Chuang added a comment - The test failure was filed in HDFS-10696 , and it was caused by HDFS-9937 .
        Hide
        eddyxu Lei (Eddy) Xu added a comment -

        Thanks for providing the patch, Manoj Govindassamy

        • The changes on FsShellPermissions.java, UmaskParser.java and ChmodParser are not relevant? Lets remove them from the patch.
        • PermissionParser.java#73, is it a format only change?
        • In PermissionParser#applyOctalPattern:
          stickyBitType = userType = groupType = othersType = '=';
          

          Should we only set stickyBitType='='?

        • TestStickyBit.java. Please clean up imports.
          // Tear down the test directories
          402	    hdfs.delete(sbExplicitTestDir, true);
          403	    hdfs.delete(sbOmittedTestDir, true);
          404	    assertFalse(hdfs.exists(sbExplicitTestDir));
          405	    assertFalse(hdfs.exists(sbOmittedTestDir));
          

          You dont need to tear down these directories. Also you might not be necessary to test the sticky pit persistent by restart the cluster, as this is covered in the other test cases.

        The rest looks good. Thanks

        Show
        eddyxu Lei (Eddy) Xu added a comment - Thanks for providing the patch, Manoj Govindassamy The changes on FsShellPermissions.java , UmaskParser.java and ChmodParser are not relevant? Lets remove them from the patch. PermissionParser.java#73 , is it a format only change? In PermissionParser#applyOctalPattern : stickyBitType = userType = groupType = othersType = '='; Should we only set stickyBitType='=' ? TestStickyBit.java . Please clean up imports. // Tear down the test directories 402 hdfs.delete(sbExplicitTestDir, true ); 403 hdfs.delete(sbOmittedTestDir, true ); 404 assertFalse(hdfs.exists(sbExplicitTestDir)); 405 assertFalse(hdfs.exists(sbOmittedTestDir)); You dont need to tear down these directories. Also you might not be necessary to test the sticky pit persistent by restart the cluster, as this is covered in the other test cases. The rest looks good. Thanks
        Hide
        cnauroth Chris Nauroth added a comment -

        Now the question is if we declare this a bug fix that can be backported to branch-2, or if this behavior change is too incompatible. Given that sticky bits are pretty rare in general, I think it's safe for branch-2, but would welcome other's thoughts. Anything to add Chris Nauroth?

        Andrew Wang, thanks for the notification. I agree with the proposed change, but the compatibility aspects of changes like this are always tricky to consider. In this case, the change is something that potentially weakens authorization. If a user has some automation that runs chmod on a directory, and that user expects the current behavior that sticky bit is preserved, then the effect would be to start allowing users to delete files owned by someone else. Admittedly, sticky bit usage is rare, typically only on /tmp, but I'd still be more comfortable with this as a 3.x change flagged backward-incompatible.

        Show
        cnauroth Chris Nauroth added a comment - Now the question is if we declare this a bug fix that can be backported to branch-2, or if this behavior change is too incompatible. Given that sticky bits are pretty rare in general, I think it's safe for branch-2, but would welcome other's thoughts. Anything to add Chris Nauroth? Andrew Wang , thanks for the notification. I agree with the proposed change, but the compatibility aspects of changes like this are always tricky to consider. In this case, the change is something that potentially weakens authorization. If a user has some automation that runs chmod on a directory, and that user expects the current behavior that sticky bit is preserved, then the effect would be to start allowing users to delete files owned by someone else. Admittedly, sticky bit usage is rare, typically only on /tmp, but I'd still be more comfortable with this as a 3.x change flagged backward-incompatible.
        Hide
        andrew.wang Andrew Wang added a comment -

        Got it, thanks Chris. Updating the JIRA fields as appropriate.

        Manoj, do you mind adding a little release note about how this change will affect end users? Could also consider changing the summary to say what this patch will do, rather than just what is broken.

        Show
        andrew.wang Andrew Wang added a comment - Got it, thanks Chris. Updating the JIRA fields as appropriate. Manoj, do you mind adding a little release note about how this change will affect end users? Could also consider changing the summary to say what this patch will do, rather than just what is broken.
        Hide
        manojg Manoj Govindassamy added a comment -

        Thanks for the inputs Chris Nauroth.
        Andrew Wang, sure will change the summary and add a release notes.

        Show
        manojg Manoj Govindassamy added a comment - Thanks for the inputs Chris Nauroth . Andrew Wang , sure will change the summary and add a release notes.
        Hide
        manojg Manoj Govindassamy added a comment -

        Thanks for the review Lei Eddy.

        1. I looked at few methods that are in the code path of chmod and cleaned them up as well. The changes in FsShellPermission.java. ChmodParser.java, UmaskParsser.java include usage of proper camelCase variable names, removal of unwanted lines, typos etc., But for now reverted these changes to make the patch more focussed.
        2. Yes, PermissionParser.java#73 is formatting only. No change in Matcher Groups. For now, reverted this change.
        3. In PermissionParser#applyOctalPattern: we need to assign '=' to all type as this is apply of new permission and all modes gets a new assignment.
        4. Cleaned up imports in TestStickyBit.java. I am using Intellij's organize imports tool and it reorganized a little more now like arranging them chronologically. Hope its ok.
        5. Removed the tear down section and the cluster restart in TestStickyBit.java
        6. Additionally, made few more changes to suffice checkstyle .

        Attached the v002 patch with review comments incorporated. Please take a look.

        Show
        manojg Manoj Govindassamy added a comment - Thanks for the review Lei Eddy. 1. I looked at few methods that are in the code path of chmod and cleaned them up as well. The changes in FsShellPermission.java . ChmodParser.java , UmaskParsser.java include usage of proper camelCase variable names, removal of unwanted lines, typos etc., But for now reverted these changes to make the patch more focussed. 2. Yes, PermissionParser.java#73 is formatting only. No change in Matcher Groups. For now, reverted this change. 3. In PermissionParser#applyOctalPattern: we need to assign '=' to all type as this is apply of new permission and all modes gets a new assignment. 4. Cleaned up imports in TestStickyBit.java. I am using Intellij's organize imports tool and it reorganized a little more now like arranging them chronologically. Hope its ok. 5. Removed the tear down section and the cluster restart in TestStickyBit.java 6. Additionally, made few more changes to suffice checkstyle . Attached the v002 patch with review comments incorporated. Please take a look.
        Hide
        manojg Manoj Govindassamy added a comment -

        Fixed checkstyle issues.

        Show
        manojg Manoj Govindassamy added a comment - Fixed checkstyle issues.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 13s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        0 mvndep 9m 30s Maven dependency ordering for branch
        +1 mvninstall 7m 24s trunk passed
        +1 compile 8m 12s trunk passed
        +1 checkstyle 1m 32s trunk passed
        +1 mvnsite 2m 2s trunk passed
        +1 mvneclipse 0m 25s trunk passed
        +1 findbugs 3m 27s trunk passed
        +1 javadoc 1m 51s trunk passed
        0 mvndep 0m 16s Maven dependency ordering for patch
        +1 mvninstall 1m 57s the patch passed
        +1 compile 7m 46s the patch passed
        +1 javac 7m 46s the patch passed
        +1 checkstyle 1m 34s root: The patch generated 0 new + 219 unchanged - 2 fixed = 219 total (was 221)
        +1 mvnsite 1m 57s the patch passed
        +1 mvneclipse 0m 31s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 3m 24s the patch passed
        +1 javadoc 1m 45s the patch passed
        +1 unit 8m 20s hadoop-common in the patch passed.
        -1 unit 74m 11s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 23s The patch does not generate ASF License warnings.
        137m 38s



        Reason Tests
        Failed junit tests hadoop.cli.TestHDFSCLI
          hadoop.hdfs.server.namenode.ha.TestEditLogTailer



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820608/HDFS-10689.003.patch
        JIRA Issue HDFS-10689
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux b424b1192cd7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / b43de80
        Default Java 1.8.0_101
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16226/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16226/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16226/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 9m 30s Maven dependency ordering for branch +1 mvninstall 7m 24s trunk passed +1 compile 8m 12s trunk passed +1 checkstyle 1m 32s trunk passed +1 mvnsite 2m 2s trunk passed +1 mvneclipse 0m 25s trunk passed +1 findbugs 3m 27s trunk passed +1 javadoc 1m 51s trunk passed 0 mvndep 0m 16s Maven dependency ordering for patch +1 mvninstall 1m 57s the patch passed +1 compile 7m 46s the patch passed +1 javac 7m 46s the patch passed +1 checkstyle 1m 34s root: The patch generated 0 new + 219 unchanged - 2 fixed = 219 total (was 221) +1 mvnsite 1m 57s the patch passed +1 mvneclipse 0m 31s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 24s the patch passed +1 javadoc 1m 45s the patch passed +1 unit 8m 20s hadoop-common in the patch passed. -1 unit 74m 11s hadoop-hdfs in the patch failed. +1 asflicense 0m 23s The patch does not generate ASF License warnings. 137m 38s Reason Tests Failed junit tests hadoop.cli.TestHDFSCLI   hadoop.hdfs.server.namenode.ha.TestEditLogTailer Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820608/HDFS-10689.003.patch JIRA Issue HDFS-10689 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux b424b1192cd7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / b43de80 Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/16226/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16226/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16226/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        manojg Manoj Govindassamy added a comment -

        Unit test failures are unrelated to this patch.

        Show
        manojg Manoj Govindassamy added a comment - Unit test failures are unrelated to this patch.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 17m 24s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        0 mvndep 14m 2s Maven dependency ordering for branch
        +1 mvninstall 8m 26s trunk passed
        +1 compile 8m 31s trunk passed
        +1 checkstyle 1m 48s trunk passed
        +1 mvnsite 2m 17s trunk passed
        +1 mvneclipse 0m 34s trunk passed
        +1 findbugs 3m 42s trunk passed
        +1 javadoc 2m 3s trunk passed
        0 mvndep 0m 16s Maven dependency ordering for patch
        +1 mvninstall 1m 49s the patch passed
        +1 compile 8m 36s the patch passed
        +1 javac 8m 36s the patch passed
        +1 checkstyle 1m 40s root: The patch generated 0 new + 219 unchanged - 2 fixed = 219 total (was 221)
        +1 mvnsite 2m 5s the patch passed
        +1 mvneclipse 0m 29s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 3m 46s the patch passed
        +1 javadoc 1m 57s the patch passed
        -1 unit 22m 10s hadoop-common in the patch failed.
        -1 unit 74m 44s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 24s The patch does not generate ASF License warnings.
        177m 46s



        Reason Tests
        Failed junit tests hadoop.cli.TestHDFSCLI
        Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820608/HDFS-10689.003.patch
        JIRA Issue HDFS-10689
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 56881d56a349 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / b43de80
        Default Java 1.8.0_101
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16224/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16224/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16224/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16224/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 17m 24s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 14m 2s Maven dependency ordering for branch +1 mvninstall 8m 26s trunk passed +1 compile 8m 31s trunk passed +1 checkstyle 1m 48s trunk passed +1 mvnsite 2m 17s trunk passed +1 mvneclipse 0m 34s trunk passed +1 findbugs 3m 42s trunk passed +1 javadoc 2m 3s trunk passed 0 mvndep 0m 16s Maven dependency ordering for patch +1 mvninstall 1m 49s the patch passed +1 compile 8m 36s the patch passed +1 javac 8m 36s the patch passed +1 checkstyle 1m 40s root: The patch generated 0 new + 219 unchanged - 2 fixed = 219 total (was 221) +1 mvnsite 2m 5s the patch passed +1 mvneclipse 0m 29s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 46s the patch passed +1 javadoc 1m 57s the patch passed -1 unit 22m 10s hadoop-common in the patch failed. -1 unit 74m 44s hadoop-hdfs in the patch failed. +1 asflicense 0m 24s The patch does not generate ASF License warnings. 177m 46s Reason Tests Failed junit tests hadoop.cli.TestHDFSCLI Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820608/HDFS-10689.003.patch JIRA Issue HDFS-10689 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 56881d56a349 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / b43de80 Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/16224/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16224/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16224/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16224/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        manojg Manoj Govindassamy added a comment -

        Attaching v004 patch - avoiding the usage of "*" in imports, and retaining imports old order.

        Show
        manojg Manoj Govindassamy added a comment - Attaching v004 patch - avoiding the usage of "*" in imports, and retaining imports old order.
        Hide
        eddyxu Lei (Eddy) Xu added a comment -

        +1 pending Jenkins.

        Thanks a lot for working on this, Manoj Govindassamy

        Show
        eddyxu Lei (Eddy) Xu added a comment - +1 pending Jenkins. Thanks a lot for working on this, Manoj Govindassamy
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 14s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        0 mvndep 0m 15s Maven dependency ordering for branch
        +1 mvninstall 6m 46s trunk passed
        +1 compile 7m 22s trunk passed
        +1 checkstyle 1m 30s trunk passed
        +1 mvnsite 1m 50s trunk passed
        +1 mvneclipse 0m 25s trunk passed
        +1 findbugs 3m 12s trunk passed
        +1 javadoc 1m 51s trunk passed
        0 mvndep 0m 16s Maven dependency ordering for patch
        +1 mvninstall 1m 55s the patch passed
        +1 compile 8m 51s the patch passed
        +1 javac 8m 51s the patch passed
        +1 checkstyle 1m 36s root: The patch generated 0 new + 219 unchanged - 2 fixed = 219 total (was 221)
        +1 mvnsite 2m 15s the patch passed
        +1 mvneclipse 0m 25s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 3m 25s the patch passed
        +1 javadoc 1m 51s the patch passed
        -1 unit 19m 39s hadoop-common in the patch failed.
        -1 unit 78m 35s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 22s The patch does not generate ASF License warnings.
        143m 30s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.datanode.TestLargeBlockReport
          hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes
        Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820753/HDFS-10689.004.patch
        JIRA Issue HDFS-10689
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux a7bfbddc65f7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 26de4f0
        Default Java 1.8.0_101
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16241/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16241/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16241/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16241/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 15s Maven dependency ordering for branch +1 mvninstall 6m 46s trunk passed +1 compile 7m 22s trunk passed +1 checkstyle 1m 30s trunk passed +1 mvnsite 1m 50s trunk passed +1 mvneclipse 0m 25s trunk passed +1 findbugs 3m 12s trunk passed +1 javadoc 1m 51s trunk passed 0 mvndep 0m 16s Maven dependency ordering for patch +1 mvninstall 1m 55s the patch passed +1 compile 8m 51s the patch passed +1 javac 8m 51s the patch passed +1 checkstyle 1m 36s root: The patch generated 0 new + 219 unchanged - 2 fixed = 219 total (was 221) +1 mvnsite 2m 15s the patch passed +1 mvneclipse 0m 25s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 25s the patch passed +1 javadoc 1m 51s the patch passed -1 unit 19m 39s hadoop-common in the patch failed. -1 unit 78m 35s hadoop-hdfs in the patch failed. +1 asflicense 0m 22s The patch does not generate ASF License warnings. 143m 30s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestLargeBlockReport   hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820753/HDFS-10689.004.patch JIRA Issue HDFS-10689 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux a7bfbddc65f7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 26de4f0 Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/16241/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16241/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16241/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16241/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        manojg Manoj Govindassamy added a comment -

        TestLargeBlockReport, TestDataNodeHotSwapVolumes and TestHttpServerLifecycle test failures are not related to the patch.

        Show
        manojg Manoj Govindassamy added a comment - TestLargeBlockReport, TestDataNodeHotSwapVolumes and TestHttpServerLifecycle test failures are not related to the patch.
        Hide
        eddyxu Lei (Eddy) Xu added a comment -

        Committed to trunk.

        Show
        eddyxu Lei (Eddy) Xu added a comment - Committed to trunk.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-trunk-Commit #10172 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10172/)
        HDFS-10689. Hdfs dfs chmod should reset sticky bit permission when the (lei: rev a3d0cba81071977ffc22a42f60b2ca07da25ae15)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/permission/TestStickyBit.java
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/PermissionParser.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #10172 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10172/ ) HDFS-10689 . Hdfs dfs chmod should reset sticky bit permission when the (lei: rev a3d0cba81071977ffc22a42f60b2ca07da25ae15) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/permission/TestStickyBit.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/PermissionParser.java

          People

          • Assignee:
            manojg Manoj Govindassamy
            Reporter:
            manojg Manoj Govindassamy
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development