Hadoop Common
  1. Hadoop Common
  2. HADOOP-6701

Incorrect exit codes for "dfs -chown", "dfs -chgrp"

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.19.1, 0.20.0, 0.20.1, 0.20.2
    • Fix Version/s: 0.20.3, 0.21.0
    • Component/s: fs
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      Commands chmod, chown and chgrp now returns non zero exit code and an error message on failure instead of returning zero.
    • Tags:
      yahoo 20.10

      Description

      ravi@localhost:~$ hadoop dfs -chgrp abcd /; echo $?
      chgrp: changing ownership of
      'hdfs://localhost/':org.apache.hadoop.security.AccessControlException: Permission denied
      0

      ravi@localhost:~$ hadoop dfs -chown abcd /; echo $?
      chown: changing ownership of
      'hdfs://localhost/':org.apache.hadoop.security.AccessControlException: Permission denied
      0

      ravi@localhost:~$ hadoop dfs -chmod 755 /DOESNTEXIST; echo $?
      chmod: could not get status for '/DOESNTEXIST': File does not exist: /DOESNTEXIST
      0

      -

      Exit codes for both of the above invocations should be non-zero to indicate that the command failed.

      1. HADOOP-6701-v20.patch
        11 kB
        Ravi Phulari
      2. HADOOP-6701-trunk.patch
        11 kB
        Ravi Phulari
      3. HADOOP-6701-trunk.patch
        11 kB
        Suresh Srinivas

        Issue Links

          Activity

          Hide
          Ravi Phulari added a comment -

          Attaching simple patch for fixing exit codes for -ch(mod/grp/own) commands.

          Show
          Ravi Phulari added a comment - Attaching simple patch for fixing exit codes for -ch(mod/grp/own) commands.
          Hide
          Ravi Phulari added a comment -

          Attaching patch for trunk.

          Show
          Ravi Phulari added a comment - Attaching patch for trunk.
          Hide
          Konstantin Boudnik added a comment -

          Looks like the patch is incomplete because it the case with chmod. The other two seem intact to me.

          Show
          Konstantin Boudnik added a comment - Looks like the patch is incomplete because it the case with chmod . The other two seem intact to me.
          Hide
          Ravi Phulari added a comment -

          Attaching patch to fix exit codes for fs -ch(own/mod/grp). Patch includes unit tests for theses issues.

          This patch does not fix bug for incorrect exit code when wild card input is given to the fs -ch(own/mod/grp).
          There is bug in FsShell.java which causes incorrect exit codes when wildcard inputs are given to -ch* commands.

          Bar.java
           for (int i=startIndex; i<args.length; i++) {
                Path srcPath = new Path(args[i]);
                FileSystem srcFs = srcPath.getFileSystem(getConf());
                Path[] paths = FileUtil.stat2Paths(srcFs.globStatus(srcPath), srcPath);
                for(Path path : paths) {
                  try {
          

          I will open new Jira to fix this issue.

          Show
          Ravi Phulari added a comment - Attaching patch to fix exit codes for fs -ch(own/mod/grp) . Patch includes unit tests for theses issues. This patch does not fix bug for incorrect exit code when wild card input is given to the fs -ch(own/mod/grp) . There is bug in FsShell.java which causes incorrect exit codes when wildcard inputs are given to -ch* commands. Bar.java for ( int i=startIndex; i<args.length; i++) { Path srcPath = new Path(args[i]); FileSystem srcFs = srcPath.getFileSystem(getConf()); Path[] paths = FileUtil.stat2Paths(srcFs.globStatus(srcPath), srcPath); for (Path path : paths) { try { I will open new Jira to fix this issue.
          Hide
          Ravi Phulari added a comment -

          Attaching patch for hadoop 20.s .

          Show
          Ravi Phulari added a comment - Attaching patch for hadoop 20.s .
          Hide
          Ravi Phulari added a comment -

          Patch for trunk .

          Show
          Ravi Phulari added a comment - Patch for trunk .
          Hide
          Ravi Phulari added a comment -

          This fix will change the way ch(mod/own/grp) returns exit codes.

          Failure to execute chmod, chown, chgrp commands will return non zero exit code with an error message.

          Show
          Ravi Phulari added a comment - This fix will change the way ch(mod/own/grp) returns exit codes. Failure to execute chmod, chown, chgrp commands will return non zero exit code with an error message.
          Hide
          Ravi Phulari added a comment -

          Patch for 0.20.

          Show
          Ravi Phulari added a comment - Patch for 0.20.
          Hide
          Suresh Srinivas added a comment -

          +1 for the patch.

          Show
          Suresh Srinivas added a comment - +1 for the patch.
          Hide
          Ravi Phulari added a comment -

          This patch changes exit codes returned from chmod, chgrp and chown operations

          • Old Behavior of chmod,chown,chgrp operation.
            • If operation is successful then - exit code 0 and no error message.
            • If operation is unsuccessful then - exit code 0 and error message printed.
               
              *If files does not exist and with globbed input*
              [rphulari@statepick-lm]> bin/hadoop fs -chmod 777 st*; echo $?
              0
              
              *If files does not exist without globbed input*
              [rphulari@statepick-lm]> bin/hadoop fs -chmod 777 st; echo $?
              chmod: could not get status for 'st': File does not exist: st
              0
              
              *If user does not have permission to change file-permission*
              ravi@localhost:~$ hadoop dfs -chgrp abcd /; echo $?
              chgrp: changing ownership of
              'hdfs://localhost/':org.apache.hadoop.security.AccessControlException: Permission denied
              0
              
              *If user does not have permission to change file-permission*
              ravi@localhost:~$ hadoop dfs -chown abcd /; echo $?
              chown: changing ownership of
              'hdfs://localhost/':org.apache.hadoop.security.AccessControlException: Permission denied
              0
              
              *If user has permission to change file-permission and file does not exist*
              ravi@localhost:~$ hadoop dfs -chmod 755 /DOESNTEXIST; echo $?
              chmod: could not get status for '/DOESNTEXIST': File does not exist: /DOESNTEXIST
              0
              
          • New Behavior of chmod,chown,chgrp operation.
            • If operation is successful then - exit code 0 is returned and no error message.
            • If operation is unsuccessful then - exit code 1 is returned and error message printed.
           
          *If file does not exist*
          [rphulari@lm]> bin/hadoop fs -chown rphulari:users est; echo $?
          chown: could not get status for 'est': File does not exist: est
          1
          
          *If files does not exist and with globbed input*
          [rphulari@statepick-lm]> bin/hadoop fs -chown rphulari:users est\*; echo $?
          chown: could not get status for 'est*'
          1
          
          *If file exist and operation is successful*
          [rphulari@statepick-lm]> bin/hadoop fs -chmod 700 test; echo $?
          0
          
          
          *If file/dir with glob pattern exits and operation is successful*
          [rphulari@statepick-lm]> bin/hadoop fs -chmod 777 test*; echo $?
          0
          [rphulari@statepick-lm]> bin/hadoop fs -ls 
          Found 2 items
          -rw-rw-rw-   1 admin    supergroup          0 2010-04-15 13:54 /user/rphulari/t
          drwxrwxrwx   - rphulari userss              0 2010-03-30 23:39 /user/rphulari/test
          
          

          Unchanged behavior - need to be fixed in different Jira
          Because following bug is due to Permissions code we will change it in separate Jira .

           
          *If user does not have permission to change file-permission*
          ravi@localhost:~$ hadoop dfs -chown abcd /; echo $?
          chown: changing ownership of
          'hdfs://localhost/':org.apache.hadoop.security.AccessControlException: Permission denied
          0
          
          Show
          Ravi Phulari added a comment - This patch changes exit codes returned from chmod, chgrp and chown operations Old Behavior of chmod,chown,chgrp operation. If operation is successful then - exit code 0 and no error message. If operation is unsuccessful then - exit code 0 and error message printed. *If files does not exist and with globbed input* [rphulari@statepick-lm]> bin/hadoop fs -chmod 777 st*; echo $? 0 *If files does not exist without globbed input* [rphulari@statepick-lm]> bin/hadoop fs -chmod 777 st; echo $? chmod: could not get status for 'st': File does not exist: st 0 *If user does not have permission to change file-permission* ravi@localhost:~$ hadoop dfs -chgrp abcd /; echo $? chgrp: changing ownership of 'hdfs://localhost/':org.apache.hadoop.security.AccessControlException: Permission denied 0 *If user does not have permission to change file-permission* ravi@localhost:~$ hadoop dfs -chown abcd /; echo $? chown: changing ownership of 'hdfs://localhost/':org.apache.hadoop.security.AccessControlException: Permission denied 0 *If user has permission to change file-permission and file does not exist* ravi@localhost:~$ hadoop dfs -chmod 755 /DOESNTEXIST; echo $? chmod: could not get status for '/DOESNTEXIST': File does not exist: /DOESNTEXIST 0 New Behavior of chmod,chown,chgrp operation. If operation is successful then - exit code 0 is returned and no error message. If operation is unsuccessful then - exit code 1 is returned and error message printed. *If file does not exist* [rphulari@lm]> bin/hadoop fs -chown rphulari:users est; echo $? chown: could not get status for 'est': File does not exist: est 1 *If files does not exist and with globbed input* [rphulari@statepick-lm]> bin/hadoop fs -chown rphulari:users est\*; echo $? chown: could not get status for 'est*' 1 *If file exist and operation is successful* [rphulari@statepick-lm]> bin/hadoop fs -chmod 700 test; echo $? 0 *If file/dir with glob pattern exits and operation is successful* [rphulari@statepick-lm]> bin/hadoop fs -chmod 777 test*; echo $? 0 [rphulari@statepick-lm]> bin/hadoop fs -ls Found 2 items -rw-rw-rw- 1 admin supergroup 0 2010-04-15 13:54 /user/rphulari/t drwxrwxrwx - rphulari userss 0 2010-03-30 23:39 /user/rphulari/test Unchanged behavior - need to be fixed in different Jira Because following bug is due to Permissions code we will change it in separate Jira . *If user does not have permission to change file-permission* ravi@localhost:~$ hadoop dfs -chown abcd /; echo $? chown: changing ownership of 'hdfs://localhost/':org.apache.hadoop.security.AccessControlException: Permission denied 0
          Hide
          Suresh Srinivas added a comment -

          Refreshing the patch for hudson to pickup.

          Show
          Suresh Srinivas added a comment - Refreshing the patch for hudson to pickup.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12443038/HADOOP-6701-trunk.patch
          against trunk revision 938788.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/486/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/486/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/486/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/486/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12443038/HADOOP-6701-trunk.patch against trunk revision 938788. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/486/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/486/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/486/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/486/console This message is automatically generated.
          Hide
          Suresh Srinivas added a comment -

          I committed the patch. Thank you Ravi.

          Show
          Suresh Srinivas added a comment - I committed the patch. Thank you Ravi.

            People

            • Assignee:
              Ravi Phulari
              Reporter:
              Ravi Phulari
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development