[AMBARI-19677] Storm service check failed after HOU to HDP-2.5.3 due to CNF for StormAtlasHook

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.4.0
• Fix Version/s: 2.5.0
• Component/s: ambari-server
• Assignee: Jonathan Hurley Assign to me
• Reporter: Vivek Sharma
• Created: 23/Jan/17 14:48
• Updated: 20/Mar/17 21:44
• Resolved: 23/Jan/17 16:44

Description

The extlib directory of Storm, which is managed by Ambari WRT things like Atlas hook JARs, must be updated during an upgrade to point to the correct versions of the JAR files.

STR

1. Deploy HDP-2.5.0.0 cluster with Ambari-2.5.0.0 (2.5.0.0-694) - secure cluster via Blueprint
2. Perform HOU to 2.5.3.0 and let it complete
3. Run Storm service check

Result

5821 [main] INFO  o.a.s.m.n.Login - successfully logged in.
5860 [main] INFO  o.a.s.StormSubmitter - Submitting topology WordCountid16ac3271_date242017 in distributed mode with conf {"storm.zookeeper.topology.auth.scheme":"digest","storm.zookeeper.topology.auth.payload":"-6656042462002862896:-8423155374846281190","topology.workers":3,"topology.debug":true}
6169 [main] INFO  o.a.s.StormSubmitter - Finished submitting topology: WordCountid16ac3271_date242017
6170 [main] INFO  o.a.s.StormSubmitter - Initializing the registered ISubmitterHook [org.apache.atlas.storm.hook.StormAtlasHook]
6170 [main] WARN  o.a.s.StormSubmitter - Error occurred in invoking submitter hook:[org.apache.atlas.storm.hook.StormAtlasHook] 
java.lang.ClassNotFoundException: org.apache.atlas.storm.hook.StormAtlasHook
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[?:1.8.0_112]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_112]
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) ~[?:1.8.0_112]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_112]
	at java.lang.Class.forName0(Native Method) ~[?:1.8.0_112]
	at java.lang.Class.forName(Class.java:264) ~[?:1.8.0_112]
	at org.apache.storm.StormSubmitter.invokeSubmitterHook(StormSubmitter.java:281) [storm-core-1.0.1.2.5.3.0-38.jar:1.0.1.2.5.3.0-38]
	at org.apache.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:257) [storm-core-1.0.1.2.5.3.0-38.jar:1.0.1.2.5.3.0-38]
	at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:310) [storm-core-1.0.1.2.5.3.0-38.jar:1.0.1.2.5.3.0-38]
	at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:157) [storm-core-1.0.1.2.5.3.0-38.jar:1.0.1.2.5.3.0-38]
	at storm.starter.WordCountTopology.main(WordCountTopology.java:77) [d5b85192dee911e6b1d9fa163e04609b.jar:?]
Exception in thread "main" org.apache.storm.hooks.SubmitterHookException: java.lang.ClassNotFoundException: org.apache.atlas.storm.hook.StormAtlasHook
	at org.apache.storm.StormSubmitter.invokeSubmitterHook(StormSubmitter.java:288)
	at org.apache.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:257)
	at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:310)
	at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:157)
	at storm.starter.WordCountTopology.main(WordCountTopology.java:77)
Caused by: java.lang.ClassNotFoundException: org.apache.atlas.storm.hook.StormAtlasHook
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:264)
	at org.apache.storm.StormSubmitter.invokeSubmitterHook(StormSubmitter.java:281)
	... 4 more

Command failed after 1 tries

Probably caused as the symlink for 2.5.3.0 still point to 2.5.0.0 as below:

[root@r6-mams-ambari-re2-7 storm]# ls -lahrt  /usr/hdp/2.5.0.0-1245/storm/extlib/
total 16K
drwxr-xr-x. 12 root root 4.0K Jan 20 04:51 ..
lrwxrwxrwx.  1 root root   81 Jan 20 05:14 storm-bridge-shim-0.7.0.2.5.0.0-1245.jar -> /usr/hdp/current/atlas-server/hook/storm/storm-bridge-shim-0.7.0.2.5.0.0-1245.jar
lrwxrwxrwx.  1 root root   88 Jan 20 05:14 atlas-plugin-classloader-0.7.0.2.5.0.0-1245.jar -> /usr/hdp/current/atlas-server/hook/storm/atlas-plugin-classloader-0.7.0.2.5.0.0-1245.jar
drwxr-xr-x.  2 root root 4.0K Jan 20 05:14 .
[root@r6-mams-ambari-re2-7 storm]# ls -lahrt  /usr/hdp/2.5.3.0-38/storm/extlib/
total 16K
drwxr-xr-x. 12 root root 4.0K Jan 20 06:03 ..
lrwxrwxrwx.  1 root root   81 Jan 20 07:08 storm-bridge-shim-0.7.0.2.5.0.0-1245.jar -> /usr/hdp/current/atlas-server/hook/storm/storm-bridge-shim-0.7.0.2.5.0.0-1245.jar
lrwxrwxrwx.  1 root root   88 Jan 20 07:08 atlas-plugin-classloader-0.7.0.2.5.0.0-1245.jar -> /usr/hdp/current/atlas-server/hook/storm/atlas-plugin-classloader-0.7.0.2.5.0.0-1245.jar
drwxr-xr-x.  2 root root 4.0K Jan 20 07:08 .

Issue Links

• links to: Reviewboard