[ACCUMULO-2227] Concurrent randomwalk fails when namenode dies after bulk import step

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 1.4.4
• Component/s: test
• Labels: ha randomwalk test
• Assignee: Bill Havanki
• Reporter: Bill Havanki
• Created: 20/Jan/14 20:59
• Updated: 03/Apr/14 22:05
• Resolved: 22/Jan/14 19:43

Description

Running Concurrent randomwalk under HDFS HA, if the active namenode is killed:

20 12:27:51,119 [retry.RetryInvocationHandler] WARN : Exception while invoking class org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete. Not retrying because the invoked method is not idempotent, and unable to determine whether it was invoked
java.io.IOException: Failed on local exception: java.io.IOException: Response is null.; Host Details : local host is: "slave.domain.com/10.20.200.113"; destination host is: "namenode.domain.com":8020;
...
 at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1487)
at org.apache.hadoop.hdfs.DistributedFileSystem.delete(DistributedFileSystem.java:355)
at org.apache.accumulo.server.test.randomwalk.concurrent.BulkImport.visit(BulkImport.java:140)
...
Caused by: java.io.IOException: Response is null.
at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:952)
at org.apache.hadoop.ipc.Client$Connection.run(Client.java:847)

This arises from an HDFS path delete call that cleans up from the bulk import. The test should be resilient here (and when the paths are made earlier in the test) so that the test can continue once failover has completed.

Issue Links

• is related to: HADOOP-9792 Retry the methods that are tagged @AtMostOnce along with @Idempotent