[SENTRY-1745] Bundle sentry-core-common into sentry-hdfs-dist to avoid NN failing with NoClassDefFoundError

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Fix Version/s: 2.0.0
• Labels: sentry-hdfs
• Assignee: Vamsee K. Yarlagadda
• Reporter: Vamsee K. Yarlagadda
• Created: 02/May/17 03:32
• Updated: 24/Jul/17 15:37
• Resolved: 12/May/17 23:03

Description

With the recent refactoring change SENTRY-1639, we introduced a new class SentryClientTransportConfigInterface. While trying to spin up a real cluster with this change, NN could fail to start with NoClassDefFoundError as we only put the jars under SENTRY_HOME/lib/plugins on the HDFS classpath and this new class is part of sentry-core-common which is not bundled inside plugins/.

NN Error log:

May 2, 9:27:40.722 AM	ERROR	org.apache.hadoop.hdfs.server.namenode.NameNode	
Failed to start namenode.
java.lang.NoClassDefFoundError: org/apache/sentry/core/common/transport/SentryClientTransportConfigInterface
	at org.apache.sentry.hdfs.SentryUpdater.getUpdates(SentryUpdater.java:41)
	at org.apache.sentry.hdfs.SentryAuthorizationInfo.update(SentryAuthorizationInfo.java:128)
	at org.apache.sentry.hdfs.SentryAuthorizationInfo.start(SentryAuthorizationInfo.java:212)
	at org.apache.sentry.hdfs.SentryAuthorizationProvider.start(SentryAuthorizationProvider.java:130)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startCommonServices(FSNamesystem.java:1203)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.startCommonServices(NameNode.java:706)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:692)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:844)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:823)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1547)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1615)
Caused by: java.lang.ClassNotFoundException: org.apache.sentry.core.common.transport.SentryClientTransportConfigInterface
	at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
	... 11 more

So, to avoid this we can bundle sentry-core-common as part of sentry-hdfs-dist.jar

Issue Links

• links to: Review board