Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1385

Make changes to MapReduce for the new UserGroupInformation APIs (HADOOP-6299)

        Details

        • Type: New Feature New Feature
        • Status: Closed
        • Priority: Major Major
        • Resolution: Fixed
        • Affects Version/s: None
        • Fix Version/s: 0.21.0
        • Component/s: None
        • Labels:
          None
        • Hadoop Flags:
          Incompatible change, Reviewed

          Description

          This is about moving the MapReduce code to use the new UserGroupInformation API as described in HADOOP-6299.

          1. Text File
            mr-6299.3.patch
            155 kB
            Devaraj Das
          2. Text File
            mr-6299.7.patch
            159 kB
            Devaraj Das
          3. Text File
            mr-6299.8.patch
            160 kB
            Devaraj Das
          4. Text File
            mr-6299.patch
            143 kB
            Devaraj Das

            Issue Links

            is blocked by

            New Feature - A new feature of the product, which has yet to be developed. HDFS-905 Make changes to HDFS for the new UserGroupInformation APIs (HADOOP-6299)

            • Major - Major loss of function.
            • Closed

            Improvement - An improvement or enhancement to an existing feature or task. HADOOP-6299 Use JAAS LoginContext for our login

            • Major - Major loss of function.
            • Closed

              Activity

              Ascending order - Click to sort in descending order
              Hide
              Permalink
              Devaraj Das added a comment -

              The patch for trunk

              Show
              Devaraj Das added a comment - The patch for trunk
              Hide
              Permalink
              Devaraj Das added a comment -

              The earlier patch went out of sync. Updated patch.

              Show
              Devaraj Das added a comment - The earlier patch went out of sync. Updated patch.
              Hide
              Permalink
              Owen O'Malley added a comment -

              Please add a call to UserGroupInformation.setConfiguration at the start of the JobTracker, and TaskTracker so that UGI uses the same configuration object. The Task should also use setConfiguration with the job's configuration.

              Show
              Owen O'Malley added a comment - Please add a call to UserGroupInformation.setConfiguration at the start of the JobTracker, and TaskTracker so that UGI uses the same configuration object. The Task should also use setConfiguration with the job's configuration.
              Hide
              Permalink
              Owen O'Malley added a comment -

              Please use the long form of the user name rather than the short form in spots like:

              -    JobInfo jobInfo = new JobInfo(jobId, new Text(ugi.getUserName()), 
+    JobInfo jobInfo = new JobInfo(jobId, new Text(ugi.getShortUserName()),
              Show
              Owen O'Malley added a comment - Please use the long form of the user name rather than the short form in spots like: - JobInfo jobInfo = new JobInfo(jobId, new Text(ugi.getUserName()), + JobInfo jobInfo = new JobInfo(jobId, new Text(ugi.getShortUserName()),
              Hide
              Permalink
              Owen O'Malley added a comment -

              The following code from Cluster.java and ClusterWithLinuxTaskController is a no-op:

              UserGroupInformation.getCurrentUser().doAs(...)
              Show
              Owen O'Malley added a comment - The following code from Cluster.java and ClusterWithLinuxTaskController is a no-op: UserGroupInformation.getCurrentUser().doAs(...)
              Hide
              Permalink
              Devaraj Das added a comment -

              Patch that addresses Owen's concerns. Running tests now. Running test-patch proved to be a real pain. Still trying to run it..

              Show
              Devaraj Das added a comment - Patch that addresses Owen's concerns. Running tests now. Running test-patch proved to be a real pain. Still trying to run it..
              Hide
              Permalink
              Devaraj Das added a comment -

              Tests pass with this patch.

              Show
              Devaraj Das added a comment - Tests pass with this patch.
              Hide
              Permalink
              Devaraj Das added a comment -

              Ok i could run findbugs on this patch and confirmed that it doesn't add any new warnings. I also ran javadoc and javac-warnings ones. No new warnings there too. I am sure I didn't introduce any new release audit warnings since there are no new files in the patch. Same for @author tags. It is almost impossible to run the stand-alone test-patch command, as it exists today, on this patch.

              Show
              Devaraj Das added a comment - Ok i could run findbugs on this patch and confirmed that it doesn't add any new warnings. I also ran javadoc and javac-warnings ones. No new warnings there too. I am sure I didn't introduce any new release audit warnings since there are no new files in the patch. Same for @author tags. It is almost impossible to run the stand-alone test-patch command, as it exists today, on this patch.
              Hide
              Permalink
              Owen O'Malley added a comment -

              +1

              Show
              Owen O'Malley added a comment - +1
              Hide
              Permalink
              Owen O'Malley added a comment -

              I just committed this. Thanks, Devaraj!

              Show
              Owen O'Malley added a comment - I just committed this. Thanks, Devaraj!
              Hide
              Permalink
              Hudson added a comment -

              Integrated in Hadoop-Mapreduce-trunk #221 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/221/)

              Show
              Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #221 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/221/ )
              Hide
              Permalink
              Hudson added a comment -

              Integrated in Hadoop-Mapreduce-trunk-Commit #225 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/225/)

              Show
              Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #225 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/225/ )
              Hide
              Permalink
              Hudson added a comment -

              Integrated in Hadoop-Mapreduce-trunk-Commit #255 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/255/)
              MAPREDUCE-1421. LinuxTaskController tests failing on trunk after the commit of . Contributed by Amareshwari Sriramadasu.

              Show
              Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #255 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/255/ ) MAPREDUCE-1421 . LinuxTaskController tests failing on trunk after the commit of . Contributed by Amareshwari Sriramadasu.
              Hide
              Permalink
              Hudson added a comment -

              Integrated in Hadoop-Mapreduce-trunk #248 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/248/)

              Show
              Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #248 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/248/ )

                People

                • Assignee:
                  Devaraj Das
                  Reporter:
                  Devaraj Das
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  5 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved:

                    Development