[ACCUMULO-1572] single node zookeeper failure kills connected accumulo servers

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.5.0
• Fix Version/s: 1.4.5, 1.5.1, 1.6.0
• Component/s: master, tserver
• Assignee: Eric C. Newton
• Reporter: Eric C. Newton
• Created: 16/Jul/13 13:38
• Updated: 22/Oct/13 17:13
• Resolved: 22/Oct/13 17:13

Description

Drew Thornton writes on the user mailing list:

If one zookeeper node is shutdown/fails/whatever and the rest of the ensemble stays up, the tablet servers attached as clients to the shutdown node immediately fail. If one of the clients happens to be the master, the cluster goes down.

Accumulo does not seem to be failing over to the remaining zookeeper nodes, and this causes me to restart the individual tablet servers again.

The zookeeper ensemble is very stable and has plenty of bandwidth/memory/processing, so taking one node down out of five doesn't crash the zookeepers, just the tablet servers...

Issue Links

• relates to: ACCUMULO-715 use curator

Sub-Tasks

• loggers die when the zookeeper they are using disconnects: Resolved
• Ensure ZooLock.watchParent is properly set in case of Connection loss: Resolved