Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2058

FairScheduler:NullPointerException in web interface when JobTracker not initialized

    Details

    • Type: Bug Bug
    • Status: Patch Available
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.22.0, 1.0.4
    • Fix Version/s: None
    • Component/s: contrib/fair-share
    • Labels:
      None

      Description

      When I contact the jobtracker web interface prior to the job tracker being fully initialized (say, if hdfs is still in safe mode), I get the following error:

      10/09/09 18:06:02 ERROR mortbay.log: /jobtracker.jsp
      java.lang.NullPointerException
      at org.apache.hadoop.mapred.FairScheduler.getJobs(FairScheduler.java:909)
      at org.apache.hadoop.mapred.JobTracker.getJobsFromQueue(JobTracker.java:4357)
      at org.apache.hadoop.mapred.JobTracker.getQueueInfoArray(JobTracker.java:4334)
      at org.apache.hadoop.mapred.JobTracker.getRootQueues(JobTracker.java:4295)
      at org.apache.hadoop.mapred.jobtracker_jsp.generateSummaryTable(jobtracker_jsp.java:44)
      at org.apache.hadoop.mapred.jobtracker_jsp._jspService(jobtracker_jsp.java:176)
      at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
      at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502)
      at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1124)
      at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:857)
      at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1115)
      at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:361)
      at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
      at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
      at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
      at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417)
      at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
      at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
      at org.mortbay.jetty.Server.handle(Server.java:324)
      at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:864)
      at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:533) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:207)
      at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:403)
      at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
      at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:522)

      1. MAPREDUCE-2058-branch-1.patch
        0.8 kB
        Gera Shegalov
      2. MAPREDUCE-2058.patch
        0.9 kB
        Kenji Kikushima

        Activity

        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12555264/MAPREDUCE-2058-branch-1.patch
        against trunk revision .

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3076//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12555264/MAPREDUCE-2058-branch-1.patch against trunk revision . -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3076//console This message is automatically generated.
        Hide
        Gera Shegalov added a comment -

        web threads have to be synchronized with the initialization otherwise there is no proper happens-before.

        Show
        Gera Shegalov added a comment - web threads have to be synchronized with the initialization otherwise there is no proper happens-before.
        Hide
        Kenji Kikushima added a comment -

        FairScheduler has 'initialized' flag.
        It become true when start() has called. (and initialized.)
        So, I changed to return empty collection when FairScheduler not initialized.

        Show
        Kenji Kikushima added a comment - FairScheduler has 'initialized' flag. It become true when start() has called. (and initialized.) So, I changed to return empty collection when FairScheduler not initialized.
        Hide
        Scott Chen added a comment -

        Quickly looked at the code. JobTracker.infoServer is started in the constructor of JobTracker.
        But FairScheduler.start() which initialize FairScheduler.poolMgr is called in JobTracker.offerService().
        So in this period if someone visite the page the NPE will be thrown.

        Show
        Scott Chen added a comment - Quickly looked at the code. JobTracker.infoServer is started in the constructor of JobTracker. But FairScheduler.start() which initialize FairScheduler.poolMgr is called in JobTracker.offerService(). So in this period if someone visite the page the NPE will be thrown.
        Hide
        Dan Adkins added a comment -

        I have observed this in https://svn.apache.org/repos/asf/hadoop/mapreduce/trunk@994941 as well as in the recent hadoop-0.21.0 release.

        Show
        Dan Adkins added a comment - I have observed this in https://svn.apache.org/repos/asf/hadoop/mapreduce/trunk@994941 as well as in the recent hadoop-0.21.0 release.
        Hide
        Todd Lipcon added a comment -

        Can you confirm the specific version this is on?

        Show
        Todd Lipcon added a comment - Can you confirm the specific version this is on?

          People

          • Assignee:
            Unassigned
            Reporter:
            Dan Adkins
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:

              Development