Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-9965

Fix NodeManager failing to start on subsequent times when Hdfs Auxillary Jar is set

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.2.0
    • Fix Version/s: 3.3.0
    • Component/s: auxservices, nodemanager
    • Labels:
      None

      Description

      Loading an auxiliary jar from a Hdfs location on a node manager works as expected on first time. The subsequent restart fails with ClassNotFoundException

      2019-11-08 03:59:49,256 INFO org.apache.hadoop.util.ApplicationClassLoader: classpath: []
      2019-11-08 03:59:49,256 INFO org.apache.hadoop.util.ApplicationClassLoader: system classes: [java., javax.accessibility., javax.activation., javax.activity., javax.annotation., javax.annotation.processing., javax.crypto., javax.imageio., javax.jws., javax.lang.model., -javax.management.j2ee., javax.management., javax.naming., javax.net., javax.print., javax.rmi., javax.script., -javax.security.auth.message., javax.security.auth., javax.security.cert., javax.security.sasl., javax.sound., javax.sql., javax.swing., javax.tools., javax.transaction., -javax.xml.registry., -javax.xml.rpc., javax.xml., org.w3c.dom., org.xml.sax., org.apache.commons.logging., org.apache.log4j., -org.apache.hadoop.hbase., org.apache.hadoop., core-default.xml, hdfs-default.xml, mapred-default.xml, yarn-default.xml]
      2019-11-08 03:59:49,257 INFO org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices failed in state INITED
      java.lang.ClassNotFoundException: org.apache.auxtest.AuxServiceFromHDFS
      	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
      	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
      	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
      	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
      	at org.apache.hadoop.util.ApplicationClassLoader.loadClass(ApplicationClassLoader.java:189)
      	at org.apache.hadoop.util.ApplicationClassLoader.loadClass(ApplicationClassLoader.java:157)
      	at java.lang.Class.forName0(Native Method)
      	at java.lang.Class.forName(Class.java:348)
      	at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxiliaryServiceWithCustomClassLoader.getInstance(AuxiliaryServiceWithCustomClassLoader.java:169)
      	at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:270)
      	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
      	at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
      	at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:321)
      	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
      	at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
      	at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:478)
      	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
      	at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:936)
      	at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:1016)
      

       

      The issue happens when reusing the previous localized auxillary service jar. The localized jar file is appended with /* when reusing which has caused the issue.

       

       

        Attachments

        1. YARN-9965-001.patch
          2 kB
          Prabhu Joseph
        2. YARN-9965-addendum-01.patch
          5 kB
          Prabhu Joseph

        Issue Links

          Activity

            People

            • Assignee:
              prabhujoseph Prabhu Joseph
              Reporter:
              prabhujoseph Prabhu Joseph

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment