Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.22.0
    • Component/s: jobtracker
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      On my local machine, JobTracker is not coming up with current trunk. Logs show the following NPE:

      2010-08-03 14:01:41,449 FATAL org.apache.hadoop.mapred.JobTracker: java.lang.NullPointerException
      at org.apache.hadoop.security.UserGroupInformation.isLoginKeytabBased(UserGroupInformation.java:703)
      at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:1383)
      at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:275)
      at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:267)
      at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:262)
      at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:4236)

      2010-08-03 14:01:41,449 INFO org.apache.hadoop.mapred.JobTracker: SHUTDOWN_MSG:

        Issue Links

          Activity

          Hide
          Ravi Gummadi added a comment -

          Is this related to MAPREDUCE-1945 which added this call to UserGroupInformation.isLoginKeytabBased() recently ?

          Show
          Ravi Gummadi added a comment - Is this related to MAPREDUCE-1945 which added this call to UserGroupInformation.isLoginKeytabBased() recently ?
          Hide
          Kan Zhang added a comment -

          Attach a patch. Can't run tests on it since trunk is broken.

          Show
          Kan Zhang added a comment - Attach a patch. Can't run tests on it since trunk is broken.
          Hide
          Kan Zhang added a comment -

          > Is this related to MAPREDUCE-1945

          Yes. I manually verified the patch on a secure cluster, but forgot to check when security is turned off.

          Show
          Kan Zhang added a comment - > Is this related to MAPREDUCE-1945 Yes. I manually verified the patch on a secure cluster, but forgot to check when security is turned off.
          Hide
          Jakob Homan added a comment -

          +1. This should work for both secure and unsecure.

          Show
          Jakob Homan added a comment - +1. This should work for both secure and unsecure.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12451174/m1992-02.patch
          against trunk revision 982087.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/349/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/349/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/349/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/349/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12451174/m1992-02.patch against trunk revision 982087. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/349/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/349/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/349/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/349/console This message is automatically generated.
          Hide
          Amareshwari Sriramadasu added a comment -

          Resubmitting for hudson as there are many test failures because of NoClassDefFoundError

          Show
          Amareshwari Sriramadasu added a comment - Resubmitting for hudson as there are many test failures because of NoClassDefFoundError
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12451174/m1992-02.patch
          against trunk revision 982242.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/351/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/351/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/351/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/351/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12451174/m1992-02.patch against trunk revision 982242. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/351/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/351/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/351/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/351/console This message is automatically generated.
          Hide
          Kan Zhang added a comment -

          This patch has been manually verified. Simply bring up a jobtracker with security turned off and check it's up properly and no NPE is thrown.

          Show
          Kan Zhang added a comment - This patch has been manually verified. Simply bring up a jobtracker with security turned off and check it's up properly and no NPE is thrown.
          Hide
          Devaraj Das added a comment -

          Although the patch solves the issue at hand, I will like to understand how the testcases, that all have unsecure setup, are passing without the patch. Kan, could you please dig a bit deeper and get the explanation. Thanks!

          Show
          Devaraj Das added a comment - Although the patch solves the issue at hand, I will like to understand how the testcases, that all have unsecure setup, are passing without the patch. Kan, could you please dig a bit deeper and get the explanation. Thanks!
          Hide
          Kan Zhang added a comment -

          I wonder about that, too. But I suggest we commit this patch now so that other people are unblocked. I'll take a look later on why existing unit tests didn't catch it and open a new jira if necessary.

          Show
          Kan Zhang added a comment - I wonder about that, too. But I suggest we commit this patch now so that other people are unblocked. I'll take a look later on why existing unit tests didn't catch it and open a new jira if necessary.
          Hide
          Devaraj Das added a comment -

          Okay. I have committed the fix to trunk (since the fix seems the right thing to do anyway) but please investigate why the unit tests didn't catch the problem earlier. I am not marking the issue "resolved" yet.

          Show
          Devaraj Das added a comment - Okay. I have committed the fix to trunk (since the fix seems the right thing to do anyway) but please investigate why the unit tests didn't catch the problem earlier. I am not marking the issue "resolved" yet.
          Hide
          Kan Zhang added a comment -

          Looks like I screwed it up when I ran tests for MAPREDUCE-1945, since the Hudson run for MAPREDUCE-1993 clearly caught this bug. I ran my local test again and caught it, too.

          Show
          Kan Zhang added a comment - Looks like I screwed it up when I ran tests for MAPREDUCE-1945 , since the Hudson run for MAPREDUCE-1993 clearly caught this bug. I ran my local test again and caught it, too.
          Hide
          Chris Douglas added a comment -

          Resolving, as the patch has been committed.

          Show
          Chris Douglas added a comment - Resolving, as the patch has been committed.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #523 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/523/)

          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #523 (See https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/523/ )

            People

            • Assignee:
              Kan Zhang
              Reporter:
              Ravi Gummadi
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development