[ACCUMULO-3421] DistributedTrace.enable will eat exceptions about failing to connect to ZK

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.7.0
• Component/s: test
• Assignee: Josh Elser
• Reporter: Josh Elser
• Created: 15/Dec/14 16:53
• Updated: 02/Jan/15 21:37
• Resolved: 16/Dec/14 20:04

Description

From a failed TracerRecoversAfterOfflineTableIT

java.lang.RuntimeException: Failed to connect to zookeeper (localhost:2181) within 2x zookeeper timeout period 30000
	at org.apache.accumulo.fate.zookeeper.ZooSession.connect(ZooSession.java:118)
	at org.apache.accumulo.fate.zookeeper.ZooSession.getSession(ZooSession.java:163)
	at org.apache.accumulo.fate.zookeeper.ZooReader.getSession(ZooReader.java:39)
	at org.apache.accumulo.fate.zookeeper.ZooReader.getZooKeeper(ZooReader.java:43)
	at org.apache.accumulo.fate.zookeeper.ZooReader.exists(ZooReader.java:166)
	at org.apache.accumulo.tracer.ZooTraceClient.process(ZooTraceClient.java:82)
	at org.apache.accumulo.tracer.ZooTraceClient.configure(ZooTraceClient.java:75)
	at org.apache.accumulo.core.trace.DistributedTrace.loadInstance(DistributedTrace.java:184)
	at org.apache.accumulo.core.trace.DistributedTrace.loadSpanReceivers(DistributedTrace.java:166)
	at org.apache.accumulo.core.trace.DistributedTrace.enableTracing(DistributedTrace.java:143)
	at org.apache.accumulo.core.trace.DistributedTrace.enable(DistributedTrace.java:101)
	at org.apache.accumulo.core.trace.DistributedTrace.enable(DistributedTrace.java:86)
	at org.apache.accumulo.test.TracerRecoversAfterOfflineTableIT.test(TracerRecoversAfterOfflineTableIT.java:77)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)

The problem is that org.apache.accumulo.tracer.ZooTraceClient.process(ZooTraceClient.java:82) eats the Exception and it doesn't propagate back up the stack through org.apache.accumulo.tracer.ZooTraceClient.configure(ZooTraceClient.java:75). Thus, the test just saw a "successful" call to DistributedTrace.enable, tried to run the test, which ultimately failed because tracing wasn't actually enabled.

I think we need to make sure that such an exception propagates back to the caller.

Issue Links

• breaks: ACCUMULO-3461 ZooTraceClientTest is prone to a race condition
• is broken by: ACCUMULO-898 look into replacing cloudtrace