[YARN-5045] hbase unit tests fail due to dependency issues

• Type: Sub-task
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: YARN-2928
• Fix Version/s: 2.9.0, 3.0.0-alpha1
• Component/s: timelineserver
• Hadoop Flags: Reviewed
• Assignee: Sangjin Lee
• Reporter: Sangjin Lee
• Created: 05/May/16 16:44
• Updated: 21/Oct/17 06:05
• Resolved: 07/May/16 08:21

Description

After the 5/4 rebase, the hbase unit tests in the timeline service project are failing:

org.apache.hadoop.yarn.server.timelineservice.reader.TestTimelineReaderWebServicesHBaseStorage  Time elapsed: 5.103 sec  <<< ERROR!
java.io.IOException: Shutting down
	at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
	at org.apache.hadoop.hbase.http.HttpServer.addDefaultServlets(HttpServer.java:677)
	at org.apache.hadoop.hbase.http.HttpServer.initializeWebServer(HttpServer.java:546)
	at org.apache.hadoop.hbase.http.HttpServer.<init>(HttpServer.java:500)
	at org.apache.hadoop.hbase.http.HttpServer.<init>(HttpServer.java:104)
	at org.apache.hadoop.hbase.http.HttpServer$Builder.build(HttpServer.java:345)
	at org.apache.hadoop.hbase.http.InfoServer.<init>(InfoServer.java:77)
	at org.apache.hadoop.hbase.regionserver.HRegionServer.putUpWebUI(HRegionServer.java:1697)
	at org.apache.hadoop.hbase.regionserver.HRegionServer.<init>(HRegionServer.java:550)
	at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:333)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
	at org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:139)
	at org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:217)
	at org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:153)
	at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:213)
	at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:93)
	at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:978)
	at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:938)
	at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:812)
	at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:806)
	at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:750)
	at org.apache.hadoop.yarn.server.timelineservice.reader.TestTimelineReaderWebServicesHBaseStorage.setup(TestTimelineReaderWebServicesHBaseStorage.java:87)

The root cause is that the hbase mini server depends on hadoop common's MetricsServlet which has been removed in the trunk (HADOOP-12504):

Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/metrics/MetricsServlet
        at org.apache.hadoop.hbase.http.HttpServer.addDefaultServlets(HttpServer.java:677)
        at org.apache.hadoop.hbase.http.HttpServer.initializeWebServer(HttpServer.java:546)
        at org.apache.hadoop.hbase.http.HttpServer.<init>(HttpServer.java:500)
        at org.apache.hadoop.hbase.http.HttpServer.<init>(HttpServer.java:104)
        at org.apache.hadoop.hbase.http.HttpServer$Builder.build(HttpServer.java:345)
        at org.apache.hadoop.hbase.http.InfoServer.<init>(InfoServer.java:77)
        at org.apache.hadoop.hbase.regionserver.HRegionServer.putUpWebUI(HRegionServer.java:1697)
        at org.apache.hadoop.hbase.regionserver.HRegionServer.<init>(HRegionServer.java:550)
        at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:333)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
        at org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:139)
        ... 26 more

Issue Links

• is related to: HBASE-15812 HttpServer fails to come up against hadoop trunk
• is related to: HADOOP-12504 Remove metrics v1
• relates to: YARN-5071 address HBase compatibility issues with trunk