Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-563 Security features for Map/Reduce
  3. MAPREDUCE-871

Job/Task local files have incorrect group ownership set by LinuxTaskController binary

    Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: tasktracker
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Fixed LinuxTaskController binary so that permissions of local files on TT are set correctly: user owned by the job-owner and group-owned by the group owner of the binary and _not_ the primary group of the TaskTracker.

      Description

      HADOOP-4491 fixed the secure permissions of local files on a TT. While testing HADOOP-4491 on a cluster, Karam Singh found out a bug. All the files/dirs have should be owned by the group corresponding to the group owner of the task-controller binary (via using getegid()) which in turn is a special group to which only TT user belongs. HADOOP-4491 incorrectly set it to primary group of the TT via getgid(), and not the special group.

      1. 871.20S.patch
        8 kB
        Ravi Gummadi
      2. MAPREDUCE-871-20090820.1.txt
        8 kB
        Vinod Kumar Vavilapalli

        Issue Links

          Activity

          Vinod Kumar Vavilapalli created issue -
          Vinod Kumar Vavilapalli made changes -
          Field Original Value New Value
          Assignee Vinod K V [ vinodkv ]
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Here's the scenario.

          TT process user/group information: mapred group1,group2,group3 (group1 is primary)
          task-controller ownership: --Sr-s-- 1 root group2
          job submitter: userA:groupA

          The bug is the ownership of files created for the task, for e.g.

          $jobid dr-xrws--- userA group1

          Attaching patch to fix this issue which essentially sets the permissions of files created for the task as

          $jobid dr-xrws--- userA group2

          Also modified the testcase to reflect this.

          Show
          Vinod Kumar Vavilapalli added a comment - Here's the scenario. TT process user/group information: mapred group1,group2,group3 (group1 is primary) task-controller ownership: -- Sr-s -- 1 root group2 job submitter: userA:groupA The bug is the ownership of files created for the task, for e.g. $jobid dr-xrws--- userA group1 Attaching patch to fix this issue which essentially sets the permissions of files created for the task as $jobid dr-xrws--- userA group2 Also modified the testcase to reflect this.
          Vinod Kumar Vavilapalli made changes -
          Attachment MAPREDUCE-871-20090820.1.txt [ 12417121 ]
          Vinod Kumar Vavilapalli made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12417121/MAPREDUCE-871-20090820.1.txt
          against trunk revision 806152.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 6 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/498/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/498/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/498/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/498/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12417121/MAPREDUCE-871-20090820.1.txt against trunk revision 806152. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/498/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/498/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/498/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-vesta.apache.org/498/console This message is automatically generated.
          Hide
          Vinod Kumar Vavilapalli added a comment -

          All the test failures reported as Hudson says are of age >>1 and are unrelated.

          Show
          Vinod Kumar Vavilapalli added a comment - All the test failures reported as Hudson says are of age >>1 and are unrelated.
          Vinod Kumar Vavilapalli made changes -
          Link This issue blocks MAPREDUCE-856 [ MAPREDUCE-856 ]
          Hide
          Hemanth Yamijala added a comment -

          I just committed this. Thanks, Vinod !

          Show
          Hemanth Yamijala added a comment - I just committed this. Thanks, Vinod !
          Hemanth Yamijala made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Fix Version/s 0.21.0 [ 12314045 ]
          Resolution Fixed [ 1 ]
          Vinod Kumar Vavilapalli made changes -
          Release Note Fixed LinuxTaskController binary so that permissions of local files on TT are set correctly: user owned by the job-owner and group-owned by the group owner of the binary and _not_ the primary group of the TaskTracker.
          Hide
          Ravi Gummadi added a comment -

          Attaching patch for Y! 20 distribution. Not for commit here.

          Show
          Ravi Gummadi added a comment - Attaching patch for Y! 20 distribution. Not for commit here.
          Ravi Gummadi made changes -
          Attachment 871.20S.patch [ 12430867 ]
          Tom White made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Patch Available Patch Available
          6d 1h 34m 1 Vinod Kumar Vavilapalli 20/Aug/09 10:56
          Patch Available Patch Available Resolved Resolved
          7d 22m 1 Hemanth Yamijala 27/Aug/09 11:18
          Resolved Resolved Closed Closed
          362d 10h 57m 1 Tom White 24/Aug/10 22:15

            People

            • Assignee:
              Vinod Kumar Vavilapalli
              Reporter:
              Vinod Kumar Vavilapalli
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development