[OAK-2131] Reduce usage of _lastRev

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.1.3
• Component/s: core, mongomk
• Assignee: Marcel Reutegger
• Reporter: Marcel Reutegger
• Created: 25/Sep/14 07:35
• Updated: 02/Feb/17 13:09
• Resolved: 08/Dec/14 13:14

Description

As described in OAK-1768 the usage of _lastRev must be reduced to better handle large transaction. The short term solution implemented for OAK-1768 relies on MapDB to temporarily store the pending modifications to a temp file. This solution prevents an OOME when there is a large commit, but can take quite a while to update the _lastRev fields after the branch is merged. This has an impact on the visibility of changes in a cluster because any further changes done after the merge only become visible when the large update of the _lastRev fields went through.

Delayed propagation of changes can become a problem in a cluster when some components rely on a timely visibility of changes. One example is the Apache Sling Topology, which sends heartbeats through the cluster by updating properties in the repository. The topology implementation will consider a cluster node dead if those updates are not propagated in the cluster within a given timeout. Ultimately this may lead to a cluster with multiple leaders.

Issue Links

• blocks: OAK-2324 Remove dependency to MapDB
• is related to: OAK-3112 Performance degradation of UnsavedModifications on MapDB
• is related to: OAK-5557 incomplete diffManyChildren during commitHook evaluation in a persisted branch
• is related to: OAK-5337 LastRevRecoveryAgent should avoid recovering documents from its own cluster id if the instance is running
• relates to: OAK-1768 DocumentNodeBuilder.setChildNode() runs OOM with large tree
• relates to: OAK-3079 LastRevRecoveryAgent can update _lastRev of children but not the root