Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-10314

YarnClient throws NoClassDefFoundError for WebSocketException with only shaded client jars

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 3.3.0
    • Fix Version/s: 3.3.0
    • Component/s: yarn
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      After YARN-8778, with only shaded hadoop client jars in classpath Unable to submit job.

      CC: Ayush Saxena confirmed the same. Hive 4.0 doesnot work due to this, shaded client is necessary there to avoid guava jar's conflicts.

      Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/shaded/org/eclipse/jetty/websocket/api/WebSocketException
      	at org.apache.hadoop.yarn.client.api.YarnClient.createYarnClient(YarnClient.java:92)
      	at org.apache.hadoop.mapred.ResourceMgrDelegate.<init>(ResourceMgrDelegate.java:109)
      	at org.apache.hadoop.mapred.YARNRunner.<init>(YARNRunner.java:153)
      	at org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
      	at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:130)
      	at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:109)
      	at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:102)
      	at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1545)
      	at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1541)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at javax.security.auth.Subject.doAs(Subject.java:422)
      	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1845)
      	at org.apache.hadoop.mapreduce.Job.connect(Job.java:1541)
      	at org.apache.hadoop.mapreduce.Job.submit(Job.java:1570)
      	at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1594)
      	at org.apache.hadoop.examples.WordCount.main(WordCount.java:87)
      Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.shaded.org.eclipse.jetty.websocket.api.WebSocketException
      	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
      	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
      	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
      	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
      	... 16 more
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                vinayakumarb Vinayakumar B
                Reporter:
                vinayakumarb Vinayakumar B
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: