[OAK-4180] Use another NodeStore as a local cache for a remote Document store

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.5.7, 1.6.0
• Component/s: documentmk
• Labels: secondary-nodestore
• Assignee: Chetan Mehrotra
• Reporter: Chetan Mehrotra
• Created: 12/Apr/16 07:08
• Updated: 19/May/17 04:09
• Resolved: 01/Aug/16 05:50

Description

DocumentNodeStore makes use of persistent cache to speed up its processing and save on making remote calls for data already present in cache

In addition to that we can look into make use of Segment NodeStore as kind of "local copy" for certain paths in repository and route calls to it if possible. As part of this task I would like to prototype such an approach. At high level it would work as below

1. At start bootstrap the setup and shutdown it down
2. Use a modified "sidegrade" and copy over the NodeStats from Document store to Segment store. In such a copy we also store some Document specific properties like readRevision and lastRevision as hidden property in Segment NodeStates
3. In DocumentNodeStore we refactor the current code to extract a
   1. AbstractDocumentNodeState - Abase class which has some logic move out from DocumentNodeState
   2. SegmentDocumentNodeState extends above and delegate calls to a wrapped SegmentNodeState
   3. DocumentNodeState would also extend AbstractDocumentNodeState and hence delegate to some calls to parent. In this when a call comes for getChildNode it can check if that can be served by a local copy of SegmentNodeStore for given rootRevision then it delegates to that
4. For update plan is to make use of Observer which listens to changes and updates the local copy for certain configured paths.
   1. Key aspect to address here is handle the restart case where in a cluster a specific node restarts after some time then how it refreshes itself there

Usage
Following 2 OSGi configs would need to be seed

• org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService.config

  secondary=B"true"
  

• org.apache.jackrabbit.oak.plugins.document.secondary.SecondaryStoreCacheService.config

  includedPaths=[ \
    "/",
    ]
  

With these settings if DocumentNodeStoreService gets started it would pickup the cache and use it. Change includedPaths depending on paths in repository which you want to include in secondary store.

Feature Docs
http://jackrabbit.apache.org/oak/docs/nodestore/document/secondary-store.html

Issue Links

• is blocked by: OAK-4470 Remove read revision method from DocumentNodeState
• is related to: OAK-4654 Allow to mount the secondary node store as a read-only subtree
• is related to: OAK-4655 Enable configuring multiple segment nodestore instances in same setup
• is related to: OAK-6240 Sidegrade support for DocumentNodeStore to Secondary NodeStore
• is related to: OAK-5965 Support path exclusion in secondary nodestore
• is related to: OAK-4991 Persistent cache should not cache those paths which are covered by DocumentNodeStateCache
• relates to: OAK-4626 Use oak-upgrade to initialize the DocumentMK local cache nodestore
• relates to: OAK-4978 Expose maintainence related MBeans for Segment NodeStores created via factory
• relates to: OAK-5309 Supporting roles in RepositoryManager execution of maintenance tasks
• relates to: OAK-4519 Expose SegmentNodeStore as a secondary NodeStore (oak-segment-tar)
• relates to: OAK-4606 Avoid persisting rootRevision in PathFilteringDiff

Sub-Tasks

• Move NodeState decoration related classes from oak-upgrade to oak-core: Resolved
• Introduce interface for Secondary NodeStore provider: Closed
• Expose SegmentNodeStore as a secondary NodeStore: Closed