[ZOOKEEPER-4339] ZK ensemble lost quorum and lots of exceptions seen in all ZK pods

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.6.1
• Component/s: quorum
• Assignee: Unassigned
• Reporter: priya Vijay
• Created: 29/Jul/21 07:20
• Updated: 29/Jul/21 07:34

Description

Running ZK cluster in kubernetes environment. ZK ensemble has 3 nodes

ZK ensemble is in a state where it looks like quorum is lost and see bunch of I/O exceptions . ZK cluster is not recovering from this state.

Log files have exceptions "Zookeeper is not running" but on exec to pod and running command it seems ok
echo ruok | nc 127.0.0.1 2181
imok

 

attaching the logs