Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2058

FairScheduler:NullPointerException in web interface when JobTracker not initialized

    Details

    • Type: Bug
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 0.22.0, 1.0.4
    • Fix Version/s: None
    • Component/s: contrib/fair-share
    • Labels:

      Description

      When I contact the jobtracker web interface prior to the job tracker being fully initialized (say, if hdfs is still in safe mode), I get the following error:

      10/09/09 18:06:02 ERROR mortbay.log: /jobtracker.jsp
      java.lang.NullPointerException
      at org.apache.hadoop.mapred.FairScheduler.getJobs(FairScheduler.java:909)
      at org.apache.hadoop.mapred.JobTracker.getJobsFromQueue(JobTracker.java:4357)
      at org.apache.hadoop.mapred.JobTracker.getQueueInfoArray(JobTracker.java:4334)
      at org.apache.hadoop.mapred.JobTracker.getRootQueues(JobTracker.java:4295)
      at org.apache.hadoop.mapred.jobtracker_jsp.generateSummaryTable(jobtracker_jsp.java:44)
      at org.apache.hadoop.mapred.jobtracker_jsp._jspService(jobtracker_jsp.java:176)
      at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
      at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502)
      at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1124)
      at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:857)
      at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1115)
      at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:361)
      at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
      at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
      at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
      at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417)
      at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
      at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
      at org.mortbay.jetty.Server.handle(Server.java:324)
      at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:864)
      at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:533) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:207)
      at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:403)
      at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
      at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:522)

      1. MAPREDUCE-2058-branch-1.patch
        0.8 kB
        Gera Shegalov
      2. MAPREDUCE-2058.patch
        0.9 kB
        Kenji Kikushima

        Activity

        Hide
        tlipcon Todd Lipcon added a comment -

        Can you confirm the specific version this is on?

        Show
        tlipcon Todd Lipcon added a comment - Can you confirm the specific version this is on?
        Hide
        dadkins Dan Adkins added a comment -

        I have observed this in https://svn.apache.org/repos/asf/hadoop/mapreduce/trunk@994941 as well as in the recent hadoop-0.21.0 release.

        Show
        dadkins Dan Adkins added a comment - I have observed this in https://svn.apache.org/repos/asf/hadoop/mapreduce/trunk@994941 as well as in the recent hadoop-0.21.0 release.
        Hide
        schen Scott Chen added a comment -

        Quickly looked at the code. JobTracker.infoServer is started in the constructor of JobTracker.
        But FairScheduler.start() which initialize FairScheduler.poolMgr is called in JobTracker.offerService().
        So in this period if someone visite the page the NPE will be thrown.

        Show
        schen Scott Chen added a comment - Quickly looked at the code. JobTracker.infoServer is started in the constructor of JobTracker. But FairScheduler.start() which initialize FairScheduler.poolMgr is called in JobTracker.offerService(). So in this period if someone visite the page the NPE will be thrown.
        Hide
        kj-ki Kenji Kikushima added a comment -

        FairScheduler has 'initialized' flag.
        It become true when start() has called. (and initialized.)
        So, I changed to return empty collection when FairScheduler not initialized.

        Show
        kj-ki Kenji Kikushima added a comment - FairScheduler has 'initialized' flag. It become true when start() has called. (and initialized.) So, I changed to return empty collection when FairScheduler not initialized.
        Hide
        jira.shegalov Gera Shegalov added a comment -

        web threads have to be synchronized with the initialization otherwise there is no proper happens-before.

        Show
        jira.shegalov Gera Shegalov added a comment - web threads have to be synchronized with the initialization otherwise there is no proper happens-before.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12555264/MAPREDUCE-2058-branch-1.patch
        against trunk revision .

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3076//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12555264/MAPREDUCE-2058-branch-1.patch against trunk revision . -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3076//console This message is automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 patch 0m 0s The patch command could not apply the patch during dryrun.



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12555264/MAPREDUCE-2058-branch-1.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / f1a152c
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5502/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 patch 0m 0s The patch command could not apply the patch during dryrun. Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12555264/MAPREDUCE-2058-branch-1.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / f1a152c Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5502/console This message was automatically generated.

          People

          • Assignee:
            Unassigned
            Reporter:
            dadkins Dan Adkins
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:

              Development