[KNOX-943] Incompatibility with Hadoop/HBase cglib/asm dependencies

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Invalid
• Affects Version/s: 0.12.0
• Assignee: Unassigned
• Reporter: Shane Kumpf
• Created: 20/May/17 14:32
• Updated: 28/Nov/18 19:01
• Resolved: 04/Oct/17 01:12

Description

I maintain the hadoop-mini-clusters project and have been working to update to the latest knox version in HDP 2.6.

Unfortunately, I'm running into an issue with different cglib/asm dependency versions for the WebHDFS/Knox and Stargate/Knox tests that results in the following exception.

2017-05-20 08:15:54 WARN  QueuedThreadPool:610 - 
java.lang.IncompatibleClassChangeError: class org.eclipse.jetty.annotations.AnnotationParser$MyClassVisitor has interface org.objectweb.asm.ClassVisitor as super class
	at java.lang.ClassLoader.defineClass1(Native Method)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
	at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at org.eclipse.jetty.annotations.AnnotationParser.scanClass(AnnotationParser.java:974)
	at org.eclipse.jetty.annotations.AnnotationParser.parseJarEntry(AnnotationParser.java:956)
	at org.eclipse.jetty.annotations.AnnotationParser.parseJar(AnnotationParser.java:909)
	at org.eclipse.jetty.annotations.AnnotationParser.parse(AnnotationParser.java:831)
	at org.eclipse.jetty.annotations.AnnotationConfiguration$ParserTask.call(AnnotationConfiguration.java:164)
	at org.eclipse.jetty.annotations.AnnotationConfiguration$1.run(AnnotationConfiguration.java:549)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
	at java.lang.Thread.run(Thread.java:745)
2017-05-20 08:15:54 WARN  QueuedThreadPool:610 - 
java.lang.IncompatibleClassChangeError: org/eclipse/jetty/annotations/AnnotationParser$MyClassVisitor
	at org.eclipse.jetty.annotations.AnnotationParser.scanClass(AnnotationParser.java:974)
	at org.eclipse.jetty.annotations.AnnotationParser.parseJarEntry(AnnotationParser.java:956)
	at org.eclipse.jetty.annotations.AnnotationParser.parseJar(AnnotationParser.java:909)
	at org.eclipse.jetty.annotations.AnnotationParser.parse(AnnotationParser.java:831)
	at org.eclipse.jetty.annotations.AnnotationConfiguration$ParserTask.call(AnnotationConfiguration.java:164)
	at org.eclipse.jetty.annotations.AnnotationConfiguration$1.run(AnnotationConfiguration.java:549)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
	at java.lang.Thread.run(Thread.java:745)
2017-05-20 08:15:54 WARN  QueuedThreadPool:610 - 
java.lang.IncompatibleClassChangeError: org/eclipse/jetty/annotations/AnnotationParser$MyClassVisitor
	at org.eclipse.jetty.annotations.AnnotationParser.scanClass(AnnotationParser.java:974)
	at org.eclipse.jetty.annotations.AnnotationParser.parseJarEntry(AnnotationParser.java:956)
	at org.eclipse.jetty.annotations.AnnotationParser.parseJar(AnnotationParser.java:909)
	at org.eclipse.jetty.annotations.AnnotationParser.parse(AnnotationParser.java:831)
	at org.eclipse.jetty.annotations.AnnotationConfiguration$ParserTask.call(AnnotationConfiguration.java:164)
	at org.eclipse.jetty.annotations.AnnotationConfiguration$1.run(AnnotationConfiguration.java:549)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
	at java.lang.Thread.run(Thread.java:745)
2017-05-20 08:15:54 WARN  QueuedThreadPool:610 - 
java.lang.IncompatibleClassChangeError: org/eclipse/jetty/annotations/AnnotationParser$MyClassVisitor
	at org.eclipse.jetty.annotations.AnnotationParser.scanClass(AnnotationParser.java:974)
	at org.eclipse.jetty.annotations.AnnotationParser.parseJarEntry(AnnotationParser.java:956)
	at org.eclipse.jetty.annotations.AnnotationParser.parseJar(AnnotationParser.java:909)
	at org.eclipse.jetty.annotations.AnnotationParser.parse(AnnotationParser.java:831)
	at org.eclipse.jetty.annotations.AnnotationConfiguration$ParserTask.call(AnnotationConfiguration.java:164)
	at org.eclipse.jetty.annotations.AnnotationConfiguration$1.run(AnnotationConfiguration.java:549)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
	at java.lang.Thread.run(Thread.java:745)
2017-05-20 08:15:54 WARN  QueuedThreadPool:610 - 
java.lang.IncompatibleClassChangeError: org/eclipse/jetty/annotations/AnnotationParser$MyClassVisitor
	at org.eclipse.jetty.annotations.AnnotationParser.scanClass(AnnotationParser.java:974)
	at org.eclipse.jetty.annotations.AnnotationParser.parseJarEntry(AnnotationParser.java:956)
	at org.eclipse.jetty.annotations.AnnotationParser.parseJar(AnnotationParser.java:909)
	at org.eclipse.jetty.annotations.AnnotationParser.parse(AnnotationParser.java:831)
	at org.eclipse.jetty.annotations.AnnotationConfiguration$ParserTask.call(AnnotationConfiguration.java:164)
	at org.eclipse.jetty.annotations.AnnotationConfiguration$1.run(AnnotationConfiguration.java:549)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
	at java.lang.Thread.run(Thread.java:745)
2017-05-20 08:15:54 WARN  QueuedThreadPool:617 - Unexpected thread death: org.eclipse.jetty.util.thread.QueuedThreadPool$3@764f86b5 in qtp1890777616{STARTED,8<=8<=254,i=3,q=0}
2017-05-20 08:15:54 WARN  QueuedThreadPool:617 - Unexpected thread death: org.eclipse.jetty.util.thread.QueuedThreadPool$3@764f86b5 in qtp1890777616{STARTED,8<=8<=254,i=3,q=0}
2017-05-20 08:15:54 WARN  QueuedThreadPool:617 - Unexpected thread death: org.eclipse.jetty.util.thread.QueuedThreadPool$3@764f86b5 in qtp1890777616{STARTED,8<=8<=254,i=3,q=0}
2017-05-20 08:15:54 WARN  QueuedThreadPool:617 - Unexpected thread death: org.eclipse.jetty.util.thread.QueuedThreadPool$3@764f86b5 in qtp1890777616{STARTED,8<=8<=254,i=3,q=0}
2017-05-20 08:15:54 WARN  QueuedThreadPool:617 - Unexpected thread death: org.eclipse.jetty.util.thread.QueuedThreadPool$3@764f86b5 in qtp1890777616{STARTED,8<=8<=254,i=3,q=0}

I've attempted to exclude the various versions of asm/cglib, use cglib-nodep, relocate asm packages, etc. Any changes I make results in another system failing, i.e. excluding the older asm versions results in jersey failures.

Here is a branch with the all of the updates that runs into the exception above. https://github.com/sakserv/hadoop-mini-clusters/tree/hdp26

The specific test that fails is: KnoxLocalClusterIntegrationTest.

The ASM FAQ talks about how tools and frameworks should repackage ASM, and it seems many already do. http://asm.ow2.org/doc/faq.html#Q15

Any thoughts on how to address this?