Uploaded image for project: 'Jackrabbit Oak'
  1. Jackrabbit Oak
  2. OAK-6246

Support for out of band indexing with read only access to NodeStore

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.7.4, 1.8.0
    • Component/s: run
    • Labels:
      None

      Description

      Provide support for out of band indexing where oak-run is connected in read only mode with NodeStore and indexes are stored on file system. These are then imported back by target system.

      Had a discussion with Vikas Saurabh and following flow was determined

      1. Admin would create provision a checkpoint via CheckpointMBean
      2. oak-run index is connected to NodeStore in read only mode and passed with
        • checkpoint from previous step
        • list of indexes which need to be reindexed
      3. oak-run index logic would then proceed with reindexing. However the created index data would be stored locally. This would make use of
        • DirectoryFactory - OAK-6243
        • Copy-on-write nodestore approach as being used in OAK-6220
      4. Once indexing is completed it would dump all index to an output folder with some metadata
      5. Then admin can copy this index data and use an MBean on the target setup to "import" it back. This import would need to
        • Pause the current async indexers
        • Import the external index files
        • Bring the external indexer upto date to there respective lanes checkpoint
        • Resume the async indexer

      The benefit of this approach is that

      1. We only need to backport the import logic. Rest all can be implemented in trunk and need not be backported.
      2. Using read-only mode allow oak-run from trunk to be safely connected to any of the old versions

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                chetanm Chetan Mehrotra
                Reporter:
                chetanm Chetan Mehrotra
              • Votes:
                1 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: