[BIGTOP-3923] Add missing jars for Ranger caused by the component version specified

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.0
• Fix Version/s: 3.3.0
• Component/s: rpm

Environment

centos-7(x86_64)
ambari-2.8.0
bigtop-3.3.0

• Assignee: Yu Hou
• Reporter: Yu Hou
• Created: 03/Apr/23 06:54
• Updated: 19/May/23 06:46
• Resolved: 19/May/23 06:46

Description

As the following command was executed by Ambari,

java -cp '/usr/bigtop/current/ranger-usersync/lib/*' org.apache.ranger.credentialapi.buildks create ranger.usersync.policymgr.password  -value [PROTECTED]  -provider jceks://file/usr/bigtop/current/ranger-usersync/conf/ugsync.jceks

some errors were occured.

[root@ambari-agent-02 current]# java -cp '/usr/bigtop/current/ranger-usersync/lib/*' org.apache.ranger.credentialapi.buildks create ranger.usersync.policymgr.password  -value [PROTECTED]  -provider jceks://file/usr/hdp/current/ranger-usersync/conf/ugsync.jceks
Exception in thread "main" java.lang.NoClassDefFoundError: com/ctc/wstx/io/InputBootstrapper
	at org.apache.ranger.credentialapi.CredentialReader.getDecryptedString(CredentialReader.java:39)
	at org.apache.ranger.credentialapi.buildks.createCredential(buildks.java:87)
	at org.apache.ranger.credentialapi.buildks.main(buildks.java:41)
Caused by: java.lang.ClassNotFoundException: com.ctc.wstx.io.InputBootstrapper
	at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
	... 3 more

[root@ambari-agent-02 lib]# java -cp '/usr/bigtop/current/ranger-usersync/lib/*' org.apache.ranger.credentialapi.buildks create ranger.usersync.policymgr.password  -value [PROTECTED]  -provider jceks://file/usr/hdp/current/ranger-usersync/conf/ugsync.jceks
Exception in thread "main" java.lang.NoClassDefFoundError: org/codehaus/stax2/XMLInputFactory2
	at java.lang.ClassLoader.defineClass1(Native Method)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:473)
	at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
	at org.apache.hadoop.conf.Configuration.<clinit>(Configuration.java:357)
	at org.apache.ranger.credentialapi.CredentialReader.getDecryptedString(CredentialReader.java:39)
	at org.apache.ranger.credentialapi.buildks.createCredential(buildks.java:87)
	at org.apache.ranger.credentialapi.buildks.main(buildks.java:41)
Caused by: java.lang.ClassNotFoundException: org.codehaus.stax2.XMLInputFactory2
	at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
	... 16 more

[root@ambari-agent-02 ranger-usersync]# /usr/lib/jvm/java/bin/java -cp '/usr/bigtop/current/ranger-usersync/lib/*' org.apache.ranger.credentialapi.buildks create ranger.usersync.policymgr.password -value '[PROTECTED]' -provider jceks://file/usr/bigtop/current/ranger-usersync/conf/ugsync.jceks
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/thirdparty/com/google/common/base/Preconditions
	at org.apache.hadoop.conf.Configuration$DeprecationDelta.<init>(Configuration.java:430)
	at org.apache.hadoop.conf.Configuration$DeprecationDelta.<init>(Configuration.java:443)
	at org.apache.hadoop.conf.Configuration.<clinit>(Configuration.java:525)
	at org.apache.ranger.credentialapi.CredentialReader.getDecryptedString(CredentialReader.java:39)
	at org.apache.ranger.credentialapi.buildks.createCredential(buildks.java:87)
	at org.apache.ranger.credentialapi.buildks.main(buildks.java:41)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.com.google.common.base.Preconditions
	at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
	... 6 more

These errors are caused by Bigtop specifying the version of a component, such as Hadoop.

Issue Links

• Dependency: RANGER-3992 Jar files is missing when compile with higher version of Hadoop
• duplicates: BIGTOP-3912 Ranger 2.3.0 rpm packages missing some jars
• is related to: BIGTOP-3932 ranger2.4 support both in bigtop and bigtop stack