[ACCUMULO-3169] clone table failed in RW test with Internal error when table did not exist

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.6.0, 1.6.1
• Fix Version/s: 1.6.3, 1.7.1, 1.8.0
• Assignee: Christopher Tubbs
• Reporter: Keith Turner
• Created: 25/Sep/14 19:05
• Updated: 29/May/15 21:18
• Resolved: 29/May/15 21:18

Description

While running random walk against 1.6.1, I saw the following error

24 18:38:17,522 [randomwalk.Framework] ERROR: Error during random walk
java.lang.Exception: Error running node Concurrent.xml
	at org.apache.accumulo.test.randomwalk.Module.visit(Module.java:285)
	at org.apache.accumulo.test.randomwalk.Framework.run(Framework.java:63)
	at org.apache.accumulo.test.randomwalk.Framework.main(Framework.java:122)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.accumulo.start.Main$1.run(Main.java:141)
	at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.Exception: Error running node ct.CloneTable
	at org.apache.accumulo.test.randomwalk.Module.visit(Module.java:285)
	at org.apache.accumulo.test.randomwalk.Module.visit(Module.java:254)
	... 8 more
Caused by: org.apache.accumulo.core.client.AccumuloException: Internal error processing executeFateOperation
	at org.apache.accumulo.core.client.impl.TableOperationsImpl.doFateOperation(TableOperationsImpl.java:336)
	at org.apache.accumulo.core.client.impl.TableOperationsImpl.doFateOperation(TableOperationsImpl.java:293)
	at org.apache.accumulo.core.client.impl.TableOperationsImpl.doTableFateOperation(TableOperationsImpl.java:1574)
	at org.apache.accumulo.core.client.impl.TableOperationsImpl.clone(TableOperationsImpl.java:708)
	at org.apache.accumulo.test.randomwalk.concurrent.CloneTable.visit(CloneTable.java:50)
	at org.apache.accumulo.test.randomwalk.Module.visit(Module.java:254)
	... 9 more
Caused by: org.apache.thrift.TApplicationException: Internal error processing executeFateOperation
	at org.apache.thrift.TApplicationException.read(TApplicationException.java:111)
	at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:71)
	at org.apache.accumulo.core.master.thrift.FateService$Client.recv_executeFateOperation(FateService.java:144)
	at org.apache.accumulo.core.master.thrift.FateService$Client.executeFateOperation(FateService.java:125)
	at org.apache.accumulo.core.client.impl.TableOperationsImpl.executeFateOperation(TableOperationsImpl.java:249)
	at org.apache.accumulo.core.client.impl.TableOperationsImpl.doFateOperation(TableOperationsImpl.java:302)
	... 14 more

Correlated this with the following in the master log

2014-09-24 18:38:19,257 [master.MasterClientServiceHandler] ERROR: Table with id e8 does not exist
java.lang.IllegalArgumentException: Table with id e8 does not exist
        at org.apache.accumulo.core.client.impl.Tables.getNamespaceId(Tables.java:308)
        at org.apache.accumulo.master.tableOps.CloneTable.<init>(CloneTable.java:242)
        at org.apache.accumulo.master.FateServiceHandler.executeFateOperation(FateServiceHandler.java:209)
        at sun.reflect.GeneratedMethodAccessor19.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.accumulo.trace.instrument.thrift.RpcServerInvocationHandler.invoke(RpcServerInvocationHandler.java:46)
        at org.apache.accumulo.server.util.RpcWrapper$1.invoke(RpcWrapper.java:44)
        at com.sun.proxy.$Proxy17.executeFateOperation(Unknown Source)
        at org.apache.accumulo.core.master.thrift.FateService$Processor$executeFateOperation.getResult(FateService.java:450)
        at org.apache.accumulo.core.master.thrift.FateService$Processor$executeFateOperation.getResult(FateService.java:434)
        at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
        at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
        at org.apache.accumulo.server.util.TServerUtils$TimedProcessor.process(TServerUtils.java:168)
        at org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:516)
        at org.apache.accumulo.server.util.CustomNonBlockingServer$1.run(CustomNonBlockingServer.java:77)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at org.apache.accumulo.trace.instrument.TraceRunnable.run(TraceRunnable.java:47)
        at org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34)
        at java.lang.Thread.run(Thread.java:744)
2014-09-24 18:38:19,258 [thrift.ProcessFunction] ERROR: Internal error processing executeFateOperation
org.apache.thrift.TException: Table with id e8 does not exist
        at org.apache.accumulo.server.util.RpcWrapper$1.invoke(RpcWrapper.java:48)
        at com.sun.proxy.$Proxy17.executeFateOperation(Unknown Source)
        at org.apache.accumulo.core.master.thrift.FateService$Processor$executeFateOperation.getResult(FateService.java:450)
        at org.apache.accumulo.core.master.thrift.FateService$Processor$executeFateOperation.getResult(FateService.java:434)
        at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
        at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
        at org.apache.accumulo.server.util.TServerUtils$TimedProcessor.process(TServerUtils.java:168)
        at org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:516)
        at org.apache.accumulo.server.util.CustomNonBlockingServer$1.run(CustomNonBlockingServer.java:77)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at org.apache.accumulo.trace.instrument.TraceRunnable.run(TraceRunnable.java:47)
        at org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34)
        at java.lang.Thread.run(Thread.java:744)

I have not had time to look into this. My suspicion is that that a TNFE should be thrown, but is not because namespace related changes. If true then 1.5 is not impacted.

Issue Links

• is related to: ACCUMULO-3135 RW concurrent test failed with table doesn't exist exception