[HBASE-25406] [Flakey Tests] branch-2 TestShadedHBaseTestingUtility

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: flakies
• Assignee: Unassigned
• Reporter: Michael Stack
• Created: 16/Dec/20 22:11
• Updated: 16/Dec/20 22:11

Description

Odd. TestShadedHBaseTestingUtility fails reliably if run with lots of concurrency; when I run on its own, its fine. Failure says:

java.io.IOException: Shutting down
   at org.apache.hbase.shaded.TestShadedHBaseTestingUtility.setUp(TestShadedHBaseTestingUtility.java:47)
 Caused by: java.lang.RuntimeException: Failed construction of Master: class org.apache.hadoop.hbase.master.HMasterMultiple exceptions
   at org.apache.hbase.shaded.TestShadedHBaseTestingUtility.setUp(TestShadedHBaseTestingUtility.java:47)
 Caused by: java.io.IOException: Problem starting http server
   at org.apache.hbase.shaded.TestShadedHBaseTestingUtility.setUp(TestShadedHBaseTestingUtility.java:47)
 Caused by: org.apache.hbase.thirdparty.org.eclipse.jetty.util.MultiException: Multiple exceptions
   at org.apache.hbase.shaded.TestShadedHBaseTestingUtility.setUp(TestShadedHBaseTestingUtility.java:47)
 Caused by: java.lang.NoSuchMethodError: javax.servlet.ServletContext.createFilter(Ljava/lang/Class;)Ljavax/servlet/Filter;
   at org.apache.hbase.shaded.TestShadedHBaseTestingUtility.setUp(TestShadedHBaseTestingUtility.java:47)

In test run log, I see this kinda stuff:

2020-12-16 13:14:57,649 INFO  [Listener at localhost/52473] http.HttpServer (HttpServer.java:openListeners(1137)) - Jetty bound to port 52491
 2020-12-16 13:14:57,654 INFO  [Listener at localhost/52473] server.Server (Server.java:doStart(375)) - jetty-9.4.34.v20201102; built: 2020-11-02T14:15:39.302Z; git: e46af88704a893fc12cb0e3bf46e2c7b48a009e7; jvm 1.8.0_102-b14
 2020-12-16 13:14:57,765 INFO  [Listener at localhost/52473] http.HttpServer (HttpServer.java:start(1083)) - HttpServer.start() threw a MultiException
 MultiException[java.lang.NoSuchMethodError: javax.servlet.ServletContext.createFilter(Ljava/lang/Class;)Ljavax/servlet/Filter;, java.lang.NoSuchMethodError: javax.servlet.ServletContext.createFilter(Ljava/lang/Class;)Ljavax/     servlet/Filter;, java.lang.NoSuchMethodError: javax.servlet.ServletContext.createFilter(Ljava/lang/Class;)Ljavax/servlet/Filter;, java.lang.NoSuchMethodError: javax.servlet.ServletContext.createFilter(Ljava/lang/Class;)Ljavax/   servlet/Filter;]
   at org.apache.hbase.thirdparty.org.eclipse.jetty.util.MultiException.ifExceptionThrow(MultiException.java:122)
   at org.apache.hbase.thirdparty.org.eclipse.jetty.server.Server.doStart(Server.java:413)
   at org.apache.hbase.thirdparty.org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
   at org.apache.hadoop.hbase.http.HttpServer.start(HttpServer.java:1078)
   at org.apache.hadoop.hbase.http.InfoServer.start(InfoServer.java:150)
   at org.apache.hadoop.hbase.regionserver.HRegionServer.putUpWebUI(HRegionServer.java:2200)
   at org.apache.hadoop.hbase.regionserver.HRegionServer.<init>(HRegionServer.java:687)
   at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:414)
   at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
   at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
   at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
   at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
   at org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:132)
   at org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:239)
   at org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:181)
   at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:245)
   at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:115)
   at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:1177)
   at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1140)
   at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1104)
   at org.apache.hbase.shaded.TestShadedHBaseTestingUtility.setUp(TestShadedHBaseTestingUtility.java:47)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:498)
   at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
   at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
   at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
   at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33)
   at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
   at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
   at org.apache.hadoop.hbase.SystemExitRule$1.evaluate(SystemExitRule.java:38)
   at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:288)
   at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:282)
   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   at java.lang.Thread.run(Thread.java:745)
   Suppressed: java.lang.NoSuchMethodError: javax.servlet.ServletContext.createFilter(Ljava/lang/Class;)Ljavax/servlet/Filter;
....

Our ndimiduk ran into same issue when testing w/ hbase-downstreamer... (reported to me offlist).