[ACCUMULO-3296] ZooUtil.recursiveDelete looping indefinitely

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.6.2, 1.7.0
• Component/s: master
• Assignee: Josh Elser
• Reporter: Josh Elser
• Created: 04/Nov/14 19:42
• Updated: 04/Nov/14 20:41
• Resolved: 04/Nov/14 20:41

Description

ShutdownIT-shutdownDuringQuery failed.

The end of the master log had the following:

2014-11-04 09:47:56,220 [master.LiveTServerSet] INFO : Removing zookeeper lock for tserver:39492[1497a3301100002]
2014-11-04 09:47:56,243 [zookeeper.Retry] DEBUG: Sleeping for 250ms before retrying operation
2014-11-04 09:47:56,494 [zookeeper.Retry] DEBUG: Sleeping for 250ms before retrying operation
2014-11-04 09:47:56,745 [zookeeper.Retry] DEBUG: Sleeping for 250ms before retrying operation
2014-11-04 09:47:56,996 [zookeeper.Retry] DEBUG: Sleeping for 250ms before retrying operation
2014-11-04 09:47:57,247 [zookeeper.Retry] DEBUG: Sleeping for 250ms before retrying operation
2014-11-04 09:47:57,498 [zookeeper.Retry] DEBUG: Sleeping for 250ms before retrying operation
2014-11-04 09:47:57,749 [zookeeper.Retry] DEBUG: Sleeping for 250ms before retrying operation
2014-11-04 09:47:58,000 [zookeeper.Retry] DEBUG: Sleeping for 250ms before retrying operation
2014-11-04 09:47:58,252 [zookeeper.Retry] DEBUG: Sleeping for 250ms before retrying operation
2014-11-04 09:47:58,503 [zookeeper.Retry] DEBUG: Sleeping for 250ms before retrying operation
2014-11-04 09:47:58,754 [zookeeper.Retry] DEBUG: Sleeping for 250ms before retrying operation
2014-11-04 09:47:59,006 [zookeeper.Retry] DEBUG: Sleeping for 250ms before retrying operation
2014-11-04 09:47:59,257 [zookeeper.Retry] DEBUG: Sleeping for 250ms before retrying operation
2014-11-04 09:47:59,508 [zookeeper.Retry] DEBUG: Sleeping for 250ms before retrying operation
2014-11-04 09:47:59,759 [zookeeper.Retry] DEBUG: Sleeping for 250ms before retrying operation
2014-11-04 09:48:00,011 [zookeeper.Retry] DEBUG: Sleeping for 250ms before retrying operation
2014-11-04 09:48:00,262 [zookeeper.Retry] DEBUG: Sleeping for 250ms before retrying operation
2014-11-04 09:48:00,513 [zookeeper.Retry] DEBUG: Sleeping for 250ms before retrying operation

The Retry log message kept repeating until the test timed out. Every invocation of that sleep, should also include a message with the exception that was caught which caused us to perform this retry.

It seems likely that recursiveDelete isn't doing something correctly given that was the last thing the Master was about to do.