[ZOOKEEPER-2615] Zookeeper server holds onto dead/expired session ids in the watch data structures

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.4.6
• Component/s: server
• Assignee: Camille Fournier
• Reporter: guoping.gp
• Created: 14/Oct/16 04:59
• Updated: 03/Feb/22 08:50

Description

The same issue (https://issues.apache.org/jira/browse/ZOOKEEPER-1382) still can be found even with zookeeper 3.4.6.

this issue cause our production zookeeper cluster leak about 1 million watchs, after restart the server one by one, the watch count decrease to only about 40000.

I can reproduce the issue on my mac,here it is:
------------------------------------------------------------------------

pguodeMacBook-Air:bin pguo$ echo srvr | nc localhost 6181
Zookeeper version: 3.4.6-1569965, built on 02/20/2014 09:09 GMT
Latency min/avg/max: 0/1156/128513
Received: 539
Sent: 531
Connections: 1
Outstanding: 0
Zxid: 0x100000037
Mode: follower
Node count: 5
------------------------
pguodeMacBook-Air:bin pguo$ echo cons | nc localhost 6181
/127.0.0.1:55759[1](queued=0,recved=5,sent=5,sid=0x157be2732d0000e,lop=PING,est=1476372631116,to=15000,lcxid=0x1,lzxid=0xffffffffffffffff,lresp=1476372646260,llat=8,minlat=0,avglat=6,maxlat=17)
/0:0:0:0:0:0:0:1:55767[0](queued=0,recved=1,sent=0)

------------------------
pguodeMacBook-Air:bin pguo$ echo wchp | nc localhost 6181
/curator_exists_watch
0x357be48e4d90007
0x357be48e4d90009
0x157be2732d0000e

as above 4-letter's report shows, 0x357be48e4d90007 and 0x357be48e4d90009 are leaked after the two sessions expired