[HDFS-12216] Ozone: TestKeys is failing consistently

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: HDFS-7240
• Fix Version/s: HDFS-7240
• Component/s: ozone
• Target Version/s: HDFS-7240
• Hadoop Flags: Reviewed
• Assignee: Mukul Kumar Singh
• Reporter: Mukul Kumar Singh
• Created: 28/Jul/17 17:45
• Updated: 26/Apr/18 21:53
• Resolved: 19/Aug/17 08:03

Description

TestKeys and TestKeysRatis are failing consistently as noted in test logs for HDFS-12183

TestKeysRatis is failing because of the following error

2017-07-28 23:11:28,783 [StateMachineUpdater-127.0.0.1:55793] ERROR impl.StateMachineUpdater (ExitUtils.java:terminate(80)) - Terminating with exit status 2: StateMachineUpdater-127.0.0.1:55793: the StateMachineUpdater hits Throwable
org.iq80.leveldb.DBException: Closed
	at org.fusesource.leveldbjni.internal.JniDB.put(JniDB.java:123)
	at org.apache.hadoop.utils.LevelDBStore.put(LevelDBStore.java:98)
	at org.apache.hadoop.ozone.container.common.impl.KeyManagerImpl.putKey(KeyManagerImpl.java:90)
	at org.apache.hadoop.ozone.container.common.impl.Dispatcher.handlePutKey(Dispatcher.java:547)
	at org.apache.hadoop.ozone.container.common.impl.Dispatcher.keyProcessHandler(Dispatcher.java:206)
	at org.apache.hadoop.ozone.container.common.impl.Dispatcher.dispatch(Dispatcher.java:110)
	at org.apache.hadoop.ozone.container.common.transport.server.ratis.ContainerStateMachine.dispatch(ContainerStateMachine.java:94)
	at org.apache.hadoop.ozone.container.common.transport.server.ratis.ContainerStateMachine.applyTransaction(ContainerStateMachine.java:81)
	at org.apache.ratis.server.impl.RaftServerImpl.applyLogToStateMachine(RaftServerImpl.java:913)
	at org.apache.ratis.server.impl.StateMachineUpdater.run(StateMachineUpdater.java:142)
	at java.lang.Thread.run(Thread.java:745)

where as TestKeys is failing because of

2017-07-28 23:14:20,889 [Thread-486] INFO  scm.XceiverClientManager (XceiverClientManager.java:getClient(158)) - exception java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused: /127.0.0.1:55914

Issue Links

• duplicates: HDFS-12244 Ozone: the static cache provided by ContainerCache does not work in Unit tests
• duplicates: HDFS-12307 Ozone: TestKeys#testPutAndGetKeyWithDnRestart fails