Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-185

Chown , chgrp , chmod operations allowed when namenode is in safemode .

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.20.2
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Chown , chgrp , chmod operations allowed when namenode is in safemode .

      1. SafeMode-Y20.100.patch
        8 kB
        Ravi Phulari
      2. HDFS-5942.patch
        8 kB
        Ravi Phulari
      3. HDFS-185-1.patch
        8 kB
        Ravi Phulari
      4. HADOOPv20-5942.patch
        9 kB
        Ravi Phulari
      5. HADOOP-5942v2.patch
        9 kB
        Ravi Phulari

        Activity

        Hide
        Ravi Phulari added a comment -

        Attaching patch for Yahoo! Hadoop 20.100 branch.

        Show
        Ravi Phulari added a comment - Attaching patch for Yahoo! Hadoop 20.100 branch.
        Hide
        Hudson added a comment -

        Integrated in Hdfs-Patch-h5.grid.sp2.yahoo.net #148 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/148/)

        Show
        Hudson added a comment - Integrated in Hdfs-Patch-h5.grid.sp2.yahoo.net #148 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/148/ )
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #170 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk/170/)

        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #170 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk/170/ )
        Hide
        Hudson added a comment -

        Integrated in Hdfs-Patch-h2.grid.sp2.yahoo.net #83 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/83/)

        Show
        Hudson added a comment - Integrated in Hdfs-Patch-h2.grid.sp2.yahoo.net #83 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/83/ )
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #140 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/140/)
        . Disallow chown, chgrp, chmod, setQuota, and setSpaceQuota when name-node is in safemode. Contributed by Ravi Phulari.

        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #140 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/140/ ) . Disallow chown, chgrp, chmod, setQuota, and setSpaceQuota when name-node is in safemode. Contributed by Ravi Phulari.
        Hide
        Konstantin Shvachko added a comment -

        I just committed this. Thank you Ravi.

        Show
        Konstantin Shvachko added a comment - I just committed this. Thank you Ravi.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12426164/HDFS-185-1.patch
        against trunk revision 883930.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 50 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/124/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/124/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/124/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/124/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12426164/HDFS-185-1.patch against trunk revision 883930. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 50 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/124/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/124/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/124/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/124/console This message is automatically generated.
        Hide
        Ravi Phulari added a comment -

        Uploading patch created against trunk.

        This patch passed ant test-patch

         [exec] 
             [exec] 
             [exec] 
             [exec] +1 overall.  
             [exec] 
             [exec]     +1 @author.  The patch does not contain any @author tags.
             [exec] 
             [exec]     +1 tests included.  The patch appears to include 50 new or modified tests.
             [exec] 
             [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
             [exec] 
             [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
             [exec] 
             [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
             [exec] 
             [exec]     +1 release audit.  The applied patch does not increase the total number of release audit warnings.
             [exec] 
             [exec] 
             [exec] 
             [exec] 
             [exec] ======================================================================
             [exec] ======================================================================
             [exec]     Finished build.
             [exec] ======================================================================
             [exec] ======================================================================
             [exec] 
             [exec] 
        
        BUILD SUCCESSFUL
        Total time: 12 minutes 58 seconds
        
        
        Show
        Ravi Phulari added a comment - Uploading patch created against trunk. This patch passed ant test-patch [exec] [exec] [exec] [exec] +1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 50 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] [exec] [exec] [exec] ====================================================================== [exec] ====================================================================== [exec] Finished build. [exec] ====================================================================== [exec] ====================================================================== [exec] [exec] BUILD SUCCESSFUL Total time: 12 minutes 58 seconds
        Hide
        Ravi Phulari added a comment -

        Thanks for the comments and reviewing Konstantin .
        I have removed changes made in FsShellPermissions.java and updated this patch for hadoop-hdfs .
        As per our discussion I will create new Jira to fix stack trace issue .

        Show
        Ravi Phulari added a comment - Thanks for the comments and reviewing Konstantin . I have removed changes made in FsShellPermissions.java and updated this patch for hadoop-hdfs . As per our discussion I will create new Jira to fix stack trace issue .
        Hide
        Ravi Phulari added a comment -

        Attaching Patch .

        Show
        Ravi Phulari added a comment - Attaching Patch .
        Hide
        Konstantin Shvachko added a comment - - edited

        I think you do not need the changes in FsShellPermissions, where you fix one typo and introduce split("\n")[0] to getMessage().
        split() extracts the first line of the exception. But you use SubstringComparator in testConf.xml, which should match the shorter string with the longer one, right?
        The typo should be fixed separately.
        If you decide to stick to the changes in FsShellPermissions you will need to split the patch into two, since testConf.xml is in different project common.
        I meant FsShellPermissions.java is in common, not testConf.xml.

        Show
        Konstantin Shvachko added a comment - - edited I think you do not need the changes in FsShellPermissions , where you fix one typo and introduce split("\n") [0] to getMessage() . split() extracts the first line of the exception. But you use SubstringComparator in testConf.xml, which should match the shorter string with the longer one, right? The typo should be fixed separately. If you decide to stick to the changes in FsShellPermissions you will need to split the patch into two, since testConf.xml is in different project common. I meant FsShellPermissions.java is in common, not testConf.xml.
        Hide
        Ravi Phulari added a comment -

        Attaching patch created for Hadoop 20 branch.

        Show
        Ravi Phulari added a comment - Attaching patch created for Hadoop 20 branch.
        Hide
        Ravi Phulari added a comment -

        Attaching patch created for Hadoop 20 branch .

        Show
        Ravi Phulari added a comment - Attaching patch created for Hadoop 20 branch .
        Hide
        Tsz Wo Nicholas Sze added a comment -

        +1

        Show
        Tsz Wo Nicholas Sze added a comment - +1
        Hide
        Ravi Phulari added a comment -

        Contrib. test failures are not related to changes made in this patch .

        Show
        Ravi Phulari added a comment - Contrib. test failures are not related to changes made in this patch .
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12410266/HADOOP-5942v2.patch
        against trunk revision 783982.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 50 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        -1 contrib tests. The patch failed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/493/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/493/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/493/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/493/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12410266/HADOOP-5942v2.patch against trunk revision 783982. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 50 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 Eclipse classpath. The patch retains Eclipse classpath integrity. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/493/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/493/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/493/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/493/console This message is automatically generated.
        Hide
        Ravi Phulari added a comment -

        Deleted discarded patches and attached corrected updated patch fixing errors in TestDFSConf.xml .

        Show
        Ravi Phulari added a comment - Deleted discarded patches and attached corrected updated patch fixing errors in TestDFSConf.xml .
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12410018/HADOOP-5942v2.patch
        against trunk revision 782083.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 50 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed core unit tests.

        -1 contrib tests. The patch failed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/477/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/477/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/477/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/477/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12410018/HADOOP-5942v2.patch against trunk revision 782083. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 50 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 Eclipse classpath. The patch retains Eclipse classpath integrity. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/477/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/477/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/477/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/477/console This message is automatically generated.
        Hide
        Ravi Phulari added a comment -

        Thanks for comments Nicholas . Uploading new patch to as per Nicholas's comments .

        Show
        Ravi Phulari added a comment - Thanks for comments Nicholas . Uploading new patch to as per Nicholas's comments .
        Hide
        Tsz Wo Nicholas Sze added a comment -

        I think safemode checking should be done before permission checking since safemode checking is much simpler than permission.

        Show
        Tsz Wo Nicholas Sze added a comment - I think safemode checking should be done before permission checking since safemode checking is much simpler than permission.
        Hide
        Ravi Phulari added a comment - - edited

        This patch prevents execution of chown , chmod , chgrp , and quota commands when namenode is in safemode .

          [exec] 
             [exec] There appear to be 492 release audit warnings before the patch and 492 release audit warnings after applying the patch.
             [exec] 
             [exec] 
             [exec] 
             [exec] 
             [exec] +1 overall.  
             [exec] 
             [exec]     +1 @author.  The patch does not contain any @author tags.
             [exec] 
             [exec]     +1 tests included.  The patch appears to include 50 new or modified tests.
             [exec] 
             [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
             [exec] 
             [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
             [exec] 
             [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
             [exec] 
             [exec]     +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
             [exec] 
             [exec]     +1 release audit.  The applied patch does not increase the total number of release audit warnings.
             [exec] 
             [exec] 
             [exec] 
             [exec] 
             [exec] ======================================================================
             [exec] ======================================================================
             [exec]     Finished build.
             [exec] ======================================================================
             [exec] ======================================================================
             [exec] 
             [exec] 
        
        BUILD SUCCESSFUL
        Total time: 26 minutes 19 seconds
        
        
        
        Show
        Ravi Phulari added a comment - - edited This patch prevents execution of chown , chmod , chgrp , and quota commands when namenode is in safemode . [exec] [exec] There appear to be 492 release audit warnings before the patch and 492 release audit warnings after applying the patch. [exec] [exec] [exec] [exec] [exec] +1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 50 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] [exec] [exec] [exec] ====================================================================== [exec] ====================================================================== [exec] Finished build. [exec] ====================================================================== [exec] ====================================================================== [exec] [exec] BUILD SUCCESSFUL Total time: 26 minutes 19 seconds
        Hide
        Ravi Phulari added a comment -

        -setQuota and -setSpaceQuota is also allowed .

        Show
        Ravi Phulari added a comment - -setQuota and -setSpaceQuota is also allowed .
        Hide
        Ravi Phulari added a comment -

        Safe mode is a Namenode state in which it does not accept changes to the name space (read-only) .
        http://hadoop.apache.org/core/docs/r0.20.0/commands_manual.html#dfsadmin

        But when namenode is entered in safemode manually , Chown ,chgrp , chmod operations are allowed which makes changes in namespace and edit logs .

        Show
        Ravi Phulari added a comment - Safe mode is a Namenode state in which it does not accept changes to the name space (read-only) . http://hadoop.apache.org/core/docs/r0.20.0/commands_manual.html#dfsadmin But when namenode is entered in safemode manually , Chown ,chgrp , chmod operations are allowed which makes changes in namespace and edit logs .

          People

          • Assignee:
            Ravi Phulari
            Reporter:
            Ravi Phulari
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development