Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1754

Replace mapred.persmissions.supergroup with an acl : mapreduce.cluster.administrators

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.22.0
    • Component/s: jobtracker
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      mapred.permissions.supergroup should be replaced with an acl so that it does not restrict the admins to a single group.
      See more details on MAPREDUCE-1542.

      1. patch-1754.txt
        37 kB
        Amareshwari Sriramadasu
      2. patch-1754-1.txt
        36 kB
        Konstantin Shvachko
      3. patch-1754-1.txt
        36 kB
        Amareshwari Sriramadasu
      4. patch-1754-ydist.txt
        46 kB
        Amareshwari Sriramadasu

        Issue Links

          Activity

          Amareshwari Sriramadasu created issue -
          Amareshwari Sriramadasu made changes -
          Field Original Value New Value
          Link This issue is blocked by HADOOP-6748 [ HADOOP-6748 ]
          Hide
          Amareshwari Sriramadasu added a comment -

          This issue would pass mrowner followed by mapreduce.cluster.administrators value to HTTP server, when HADOOP-6748 is done.

          Show
          Amareshwari Sriramadasu added a comment - This issue would pass mrowner followed by mapreduce.cluster.administrators value to HTTP server, when HADOOP-6748 is done.
          Amareshwari Sriramadasu made changes -
          Link This issue relates to MAPREDUCE-1542 [ MAPREDUCE-1542 ]
          Hide
          Amareshwari Sriramadasu added a comment -

          Patch for Yahoo! distribution incorporating HADOOP-6748 and MAPREDUCE-1754

          Show
          Amareshwari Sriramadasu added a comment - Patch for Yahoo! distribution incorporating HADOOP-6748 and MAPREDUCE-1754
          Amareshwari Sriramadasu made changes -
          Attachment patch-1754-ydist.txt [ 12443928 ]
          Amareshwari Sriramadasu made changes -
          Link This issue is blocked by HADOOP-6862 [ HADOOP-6862 ]
          Hide
          Amareshwari Sriramadasu added a comment -

          Patch with same changes as in Y!20S patch.

          Patch does the following :
          *Removes configuration "mapred.permissions.supergroup", "mapreduce.cluster.permissions.supergroup" and "mapreduce.jobtracker.permissions.supergroup". Adds configuration "mapreduce.cluster.administrators" to specify the acl for administrators.

          • Acl specified via mapreduce.cluster.administrators and mrOwner (the user who started the mapreduce cluster) has access to all jobs, queues and common servlets.
          Show
          Amareshwari Sriramadasu added a comment - Patch with same changes as in Y!20S patch. Patch does the following : *Removes configuration "mapred.permissions.supergroup", "mapreduce.cluster.permissions.supergroup" and "mapreduce.jobtracker.permissions.supergroup". Adds configuration "mapreduce.cluster.administrators" to specify the acl for administrators. Acl specified via mapreduce.cluster.administrators and mrOwner (the user who started the mapreduce cluster) has access to all jobs, queues and common servlets.
          Amareshwari Sriramadasu made changes -
          Attachment patch-1754.txt [ 12455226 ]
          Hide
          Amareshwari Sriramadasu added a comment -

          test-patch and ant test passed on my machine.

          Show
          Amareshwari Sriramadasu added a comment - test-patch and ant test passed on my machine.
          Amareshwari Sriramadasu made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hadoop Flags [Incompatible change]
          Assignee Amareshwari Sriramadasu [ amareshwari ]
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Looked at the patch. Mostly looks good.

          • It is still possible to provide backwards compatibility for mapreduce.cluster.permissions.supergroup by simply adding the deprecated group to the new ACL list. If we do this, we should rollback the changes to ConfigUtil.java and also add one simple test.
          • TestNodeRefresh: For the change in testMRRefreshDefault(), the comment should read as // refresh by invalid user.
          Show
          Vinod Kumar Vavilapalli added a comment - Looked at the patch. Mostly looks good. It is still possible to provide backwards compatibility for mapreduce.cluster.permissions.supergroup by simply adding the deprecated group to the new ACL list. If we do this, we should rollback the changes to ConfigUtil.java and also add one simple test. TestNodeRefresh: For the change in testMRRefreshDefault(), the comment should read as // refresh by invalid user.
          Amareshwari Sriramadasu made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Amareshwari Sriramadasu added a comment -

          Patch deprecates mapreduce.cluster.permissions.supergroup instead of removing it, by adding the the group to administrator acl. Also updates the testcases.

          Show
          Amareshwari Sriramadasu added a comment - Patch deprecates mapreduce.cluster.permissions.supergroup instead of removing it, by adding the the group to administrator acl. Also updates the testcases.
          Amareshwari Sriramadasu made changes -
          Attachment patch-1754-1.txt [ 12456842 ]
          Amareshwari Sriramadasu made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hadoop Flags [Incompatible change]
          Hide
          Amareshwari Sriramadasu added a comment -

          Ran test-patch and ant test. All tests passed except the known failures on trunk.

          Show
          Amareshwari Sriramadasu added a comment - Ran test-patch and ant test. All tests passed except the known failures on trunk.
          Nigel Daley made changes -
          Fix Version/s 0.22.0 [ 12314184 ]
          Hide
          Konstantin Shvachko added a comment -

          Should we commit this to 0.22? Sounds like it's been ready for a while, and still applies to trunk.

          Show
          Konstantin Shvachko added a comment - Should we commit this to 0.22? Sounds like it's been ready for a while, and still applies to trunk.
          Hide
          Todd Lipcon added a comment -

          +1 for putting this in 0.22 if tests still pass. Can someone test-patch it?

          Show
          Todd Lipcon added a comment - +1 for putting this in 0.22 if tests still pass. Can someone test-patch it?
          Konstantin Shvachko made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Konstantin Shvachko made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Affects Version/s 0.22.0 [ 12314184 ]
          Fix Version/s 0.22.0 [ 12314184 ]
          Konstantin Shvachko made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Konstantin Shvachko made changes -
          Attachment patch-1754-1.txt [ 12469510 ]
          Hide
          Konstantin Shvachko added a comment -

          Just triggering patch build

          Show
          Konstantin Shvachko added a comment - Just triggering patch build
          Konstantin Shvachko made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Konstantin Shvachko added a comment -

          I ran tests and test-patch on this. Failed tests:
          TestTaskTrackerLocalization
          TestTrackerDistributedCacheManager

          test-patch results are below:

          .    [exec]     +1 @author.  The patch does not contain any @author tags.
               [exec] 
               [exec]     +1 tests included.  The patch appears to include 9 new or modified tests.
               [exec] 
               [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
               [exec] 
               [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
               [exec] 
               [exec]     +1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) warnings.
               [exec] 
               [exec]     +1 release audit.  The applied patch does not increase the total number of release audit warnings.
          

          Is that good to commit?

          Show
          Konstantin Shvachko added a comment - I ran tests and test-patch on this. Failed tests: TestTaskTrackerLocalization TestTrackerDistributedCacheManager test-patch results are below: . [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 9 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. Is that good to commit?
          Hide
          Konstantin Shvachko added a comment -

          I just committed this. Thank you Amareshwari.

          Show
          Konstantin Shvachko added a comment - I just committed this. Thank you Amareshwari.
          Konstantin Shvachko made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Resolution Fixed [ 1 ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #589 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/589/)
          MAPREDUCE-1754. Replace mapred.persmissions.supergroup with an acl : mapreduce.cluster.administrators. Contributed by Amareshwari Sriramadasu.

          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #589 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/589/ ) MAPREDUCE-1754 . Replace mapred.persmissions.supergroup with an acl : mapreduce.cluster.administrators. Contributed by Amareshwari Sriramadasu.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #643 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/)

          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #643 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/ )
          Konstantin Shvachko made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Patch Available Patch Available Open Open
          126d 17h 40m 3 Konstantin Shvachko 27/Jan/11 03:22
          Open Open Patch Available Patch Available
          140d 4h 4 Konstantin Shvachko 27/Jan/11 03:23
          Patch Available Patch Available Resolved Resolved
          1d 18h 12m 1 Konstantin Shvachko 28/Jan/11 21:35
          Resolved Resolved Closed Closed
          317d 8h 43m 1 Konstantin Shvachko 12/Dec/11 06:19

            People

            • Assignee:
              Amareshwari Sriramadasu
              Reporter:
              Amareshwari Sriramadasu
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development