Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5397

AM crashes because Webapp failed to start on multi node cluster

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Reopened
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      I set up a 12 nodes cluster and tried submitting jobs but get this exception.
      But job is able to succeed after AM crashes and retry a few times(2 or 3)

      2013-07-12 18:56:28,438 INFO [main] org.mortbay.log: Extract jar:file:/grid/0/dev/jhe/hadoop-2.1.0-beta/share/hadoop/yarn/hadoop-yarn-common-2.1.0-beta.jar!/webapps/mapreduce to /tmp/Jetty_0_0_0_0_43554_mapreduce____ljbmlg/webapp
      2013-07-12 18:56:28,528 WARN [main] org.mortbay.log: Failed startup of context org.mortbay.jetty.webapp.WebAppContext@2726b2{/,jar:file:/grid/0/dev/jhe/hadoop-2.1.0-beta/share/hadoop/yarn/hadoop-yarn-common-2.1.0-beta.jar!/webapps/mapreduce}
      java.io.FileNotFoundException: /tmp/Jetty_0_0_0_0_43554_mapreduce____ljbmlg/webapp/webapps/mapreduce/.keep (No such file or directory)
      	at java.io.FileOutputStream.open(Native Method)
      	at java.io.FileOutputStream.<init>(FileOutputStream.java:194)
      	at java.io.FileOutputStream.<init>(FileOutputStream.java:145)
      	at org.mortbay.resource.JarResource.extract(JarResource.java:215)
      	at org.mortbay.jetty.webapp.WebAppContext.resolveWebApp(WebAppContext.java:974)
      	at org.mortbay.jetty.webapp.WebAppContext.getWebInf(WebAppContext.java:832)
      	at org.mortbay.jetty.webapp.WebInfConfiguration.configureClassLoader(WebInfConfiguration.java:62)
      	at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:489)
      	at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
      	at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152)
      	at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156)
      	at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
      	at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
      	at org.mortbay.jetty.Server.doStart(Server.java:224)
      	at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
      	at org.apache.hadoop.http.HttpServer.start(HttpServer.java:684)
      	at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:211)
      	at org.apache.hadoop.mapreduce.v2.app.client.MRClientService.serviceStart(MRClientService.java:134)
      	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
      	at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:101)
      	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1019)
      	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
      	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1394)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at javax.security.auth.Subject.doAs(Subject.java:396)
      	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1477)
      	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1390)
      

        Attachments

        1. log.txt
          30 kB
          Jian He
        2. MRAppMasterlog.txt
          51 kB
          Rohith Sharma K S

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                jianhe Jian He
              • Votes:
                0 Vote for this issue
                Watchers:
                12 Start watching this issue

                Dates

                • Created:
                  Updated: