[OAK-5446] leaseUpdateThread might be blocked by leaseUpdateCheck

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.4, 1.5.14
• Assignee: Unassigned
• Reporter: Stefan Eissing
• Created: 12/Jan/17 09:58
• Updated: 13/May/20 07:41

Description

moved over to OAK-5528 due to internal Jira issues, please do not delete this ticket while the problem is being investigated

Fighting with cluster nodes losing their lease and shutting down oak-core in a cloud environment. For reasons unknown at this point in time, the whole process seems to skip about two minutes of real time.

This is a situation from which oak currently does not recover. Code analysis shows that ClusterNodeInfo is handed the LeaseCheckDocumentStoreWrapper instance to use as store. This is fatal since any action the renewLease() tries to do will first invoke the performLeaseCheck(). The lease check will, when the FailureMargin is reached, stall the renewLease() thread for 5 retry attempts and then declare the lease to be lost.

The ClusterNodeInfo should instead be using the "real" DocumentStore, not the wrapped one, IMO.

Issue Links

• is cloned by: OAK-5528 leaseUpdateThread might be blocked by leaseUpdateCheck
• is related to: OAK-3399 5sec retry loop before declaring lease failure (was: as a last resort try to update it there)
• is related to: OAK-5523 LeaseUpdateRetryLoop: add test coverage