When a participant or observer get partitioned and restart as Read-only server. ZkDb doesn't get reinitialized. This causes the RO server to drop any incoming request with zxid > 0
Refusing session request for client /xx.xx.xx.xx:39875
as it has seen zxid 0x2e00405fd9 our last zxid is 0x0 client must try another server
Steps to reproduce:
Start an RO-enabled observer connecting to an ensemble. Kill the ensemble and wait until the observer restart in RO mode. Zxid of this observer should be 0.
Before a server transition into LOOKING state, its database get closed as part of shutdown sequence. The database of leader, follower and observer get initialized as a side effect of participating in leader election protocol. (eg. observer will call registerWithLeader() and call getLastLoggedZxid() which initialize the db if not already).
However, RO server does not participate in this protocol so its DB doesn't get initialized properly