Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1083

Use the user-to-groups mapping service in the JobTracker

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: jobtracker
    • Labels:
      None

      Description

      HADOOP-4656 introduces a user-to-groups mapping service on the server-side. The JobTracker should use this to map users to their groups rather than relying on the information passed by the client.

      1. MR-1083-0_20.2.patch
        59 kB
        Jitendra Nath Pandey
      2. MAPREDUCE-1083-3.patch
        16 kB
        Boris Shkolnik
      3. MAPREDUCE-1083-2.patch
        16 kB
        Boris Shkolnik
      4. HADOOP-4656_mr.patch
        9 kB
        Arun C Murthy

        Issue Links

          Activity

          Hide
          Arun C Murthy added a comment -

          Preliminary patch, with some testing complete.

          Show
          Arun C Murthy added a comment - Preliminary patch, with some testing complete.
          Hide
          Boris Shkolnik added a comment -

          added and fixed tests to support common changes

          Show
          Boris Shkolnik added a comment - added and fixed tests to support common changes
          Hide
          Boris Shkolnik added a comment -

          addressed issues from Devaraj's review
          1. removed commented lines
          2. changed some System.out to LOG.info

          Show
          Boris Shkolnik added a comment - addressed issues from Devaraj's review 1. removed commented lines 2. changed some System.out to LOG.info
          Hide
          Devaraj Das added a comment -

          Submitting the patch on behalf of Boris.

          Show
          Devaraj Das added a comment - Submitting the patch on behalf of Boris.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12428377/MAPREDUCE-1083-3.patch
          against trunk revision 892178.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 5 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/220/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/220/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/220/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/220/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12428377/MAPREDUCE-1083-3.patch against trunk revision 892178. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 5 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/220/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/220/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/220/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h3.grid.sp2.yahoo.net/220/console This message is automatically generated.
          Hide
          Boris Shkolnik added a comment -

          2 contrib tests failed:
          reran them manually:

          [junit] Running org.apache.hadoop.streaming.TestStreamingExitStatus
          [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 8.194 sec

          [junit] Running org.apache.hadoop.streaming.TestStreamingKeyValue
          [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 3.185 sec

          Show
          Boris Shkolnik added a comment - 2 contrib tests failed: reran them manually: [junit] Running org.apache.hadoop.streaming.TestStreamingExitStatus [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 8.194 sec [junit] Running org.apache.hadoop.streaming.TestStreamingKeyValue [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 3.185 sec
          Hide
          Devaraj Das added a comment -

          +1. I just committed this. Thanks, Boris!

          Show
          Devaraj Das added a comment - +1. I just committed this. Thanks, Boris!
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #196 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/196/)

          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #196 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/196/ )
          Hide
          Jitendra Nath Pandey added a comment -

          Combined patch for HADOOP-4656, HDFS-685 and MAPREDUCE-1083 for Hadoop-20.

          Show
          Jitendra Nath Pandey added a comment - Combined patch for HADOOP-4656 , HDFS-685 and MAPREDUCE-1083 for Hadoop-20.

            People

            • Assignee:
              Boris Shkolnik
              Reporter:
              Arun C Murthy
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development