[HBASE-23142] ZooKeeper-Jute missing from HBOSS shaded dependencies with ZK 3.5

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: hbase-filesystem-1.0.0-alpha2
• Component/s: hboss
• Hadoop Flags: Reviewed
• Assignee: Josh Elser
• Reporter: Josh Elser
• Created: 09/Oct/19 16:16
• Updated: 09/Oct/19 17:19
• Resolved: 09/Oct/19 17:19

Description

ZooKeeper 3.5 has a transitive dependency on a zookeeper-jute artifact. If this isn't on the classpath by some other means, you'll get an error similar to:

Failed construction RegionServer
java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/oss/thirdparty/org/apache/jute/Record
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:264)
	at org.apache.hadoop.hbase.oss.thirdparty.org.apache.curator.utils.Compatibility.<clinit>(Compatibility.java:35)
	at org.apache.hadoop.hbase.oss.thirdparty.org.apache.curator.framework.CuratorFrameworkFactory$Builder.<init>(CuratorFrameworkFactory.java:149)
	at org.apache.hadoop.hbase.oss.thirdparty.org.apache.curator.framework.CuratorFrameworkFactory$Builder.<init>(CuratorFrameworkFactory.java:130)
	at org.apache.hadoop.hbase.oss.thirdparty.org.apache.curator.framework.CuratorFrameworkFactory.builder(CuratorFrameworkFactory.java:78)
	at org.apache.hadoop.hbase.oss.thirdparty.org.apache.curator.framework.CuratorFrameworkFactory.newClient(CuratorFrameworkFactory.java:104)
	at org.apache.hadoop.hbase.oss.thirdparty.org.apache.curator.framework.CuratorFrameworkFactory.newClient(CuratorFrameworkFactory.java:90)
	at org.apache.hadoop.hbase.oss.sync.ZKTreeLockManager.initialize(ZKTreeLockManager.java:93)
	at org.apache.hadoop.hbase.oss.sync.TreeLockManager.get(TreeLockManager.java:72)
	at org.apache.hadoop.hbase.oss.HBaseObjectStoreSemantics.initialize(HBaseObjectStoreSemantics.java:122)
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3315)
	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:136)
	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3364)
	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3332)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:491)
	at org.apache.hadoop.fs.Path.getFileSystem(Path.java:361)
	at org.apache.hadoop.hbase.util.CommonFSUtils.getRootDir(CommonFSUtils.java:361)
	at org.apache.hadoop.hbase.util.CommonFSUtils.isValidWALRootDir(CommonFSUtils.java:411)
	at org.apache.hadoop.hbase.util.CommonFSUtils.getWALRootDir(CommonFSUtils.java:387)
	at org.apache.hadoop.hbase.regionserver.HRegionServer.initializeFileSystem(HRegionServer.java:731)
	at org.apache.hadoop.hbase.regionserver.HRegionServer.<init>(HRegionServer.java:637)
	at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:493)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2905)
	at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:236)
	at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:140)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
	at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:149)
	at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2923)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.oss.thirdparty.org.apache.jute.Record
	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 33 more

when trying to start up any HBase service.

Just need to the shade-plugin execution to include zookeeper-jute, too. I think Maven will complain (but not error) with the default ZooKeeper 3.4 dependency.

Issue Links

• links to: GitHub Pull Request #10