Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4195

With invalid queueName request param, jobqueue_details.jsp shows NPE

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 1.0.0
    • 1.2.0
    • jobtracker
    • None
    • Reviewed

    Description

      When you access /jobqueue_details.jsp manually, instead of via a link, it has queueName set to null internally and this goes for a lookup into the scheduling info maps as well.

      As a result, if using FairScheduler, a Pool with String name = null gets created and this brings the scheduler down. I have not tested what happens to the CapacityScheduler, but ideally if no queueName is set in that jsp, it should fall back to 'default'. Otherwise, this brings down the JobTracker completely.

      FairScheduler must also add a check to not create a pool with 'null' name.

      The following is the strace that ensues:

      ERROR org.mortbay.log: /jobqueue_details.jsp 
      java.lang.NullPointerException 
      at org.apache.hadoop.mapred.jobqueue_005fdetails_jsp._jspService(jobqueue_005fdetails_jsp.java:71) 
      at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97) 
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) 
      at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) 
      at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221) 
      at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:829) 
      at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) 
      at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399) 
      at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) 
      at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) 
      at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) 
      at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) 
      at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) 
      at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) 
      at org.mortbay.jetty.Server.handle(Server.java:326) 
      at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) 
      at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928) 
      at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549) 
      at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) 
      at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) 
      at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410) 
      at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) 
      INFO org.apache.hadoop.ipc.Server: IPC Server handler 2 on 9001, call heartbeat from XYZ:MNOP: error: java.io.IOException: java.lang.NullPointerException 
      java.io.IOException: java.lang.NullPointerException 
      at org.apache.hadoop.mapred.SchedulingAlgorithms$FairShareComparator.compare(SchedulingAlgorithms.java:95) 
      at org.apache.hadoop.mapred.SchedulingAlgorithms$FairShareComparator.compare(SchedulingAlgorithms.java:68) 
      at java.util.Arrays.mergeSort(Unknown Source) 
      at java.util.Arrays.sort(Unknown Source) 
      at java.util.Collections.sort(Unknown Source) 
      at org.apache.hadoop.mapred.FairScheduler.assignTasks(FairScheduler.java:435) 
      at org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:3226) 
      at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) 
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) 
      at java.lang.reflect.Method.invoke(Unknown Source) 
      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557) 
      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434) 
      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430) 
      at java.security.AccessController.doPrivileged(Native Method) 
      at javax.security.auth.Subject.doAs(Unknown Source) 
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127) 
      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
      

      Attachments

        1. MR-4195
          0.6 kB
          Gera Shegalov
        2. MR-4195
          0.6 kB
          Gera Shegalov

        Issue Links

          Activity

            People

              Unassigned Unassigned
              jira.shegalov Gera Shegalov
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: