Uploaded image for project: 'Jackrabbit Oak'
  1. Jackrabbit Oak
  2. OAK-6246

Support for out of band indexing with read only access to NodeStore

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 1.7.4, 1.8.0
    • run
    • None

    Description

      Provide support for out of band indexing where oak-run is connected in read only mode with NodeStore and indexes are stored on file system. These are then imported back by target system.

      Had a discussion with catholicon and following flow was determined

      1. Admin would create provision a checkpoint via CheckpointMBean
      2. oak-run index is connected to NodeStore in read only mode and passed with
        • checkpoint from previous step
        • list of indexes which need to be reindexed
      3. oak-run index logic would then proceed with reindexing. However the created index data would be stored locally. This would make use of
        • DirectoryFactory - OAK-6243
        • Copy-on-write nodestore approach as being used in OAK-6220
      4. Once indexing is completed it would dump all index to an output folder with some metadata
      5. Then admin can copy this index data and use an MBean on the target setup to "import" it back. This import would need to
        • Pause the current async indexers
        • Import the external index files
        • Bring the external indexer upto date to there respective lanes checkpoint
        • Resume the async indexer

      The benefit of this approach is that

      1. We only need to backport the import logic. Rest all can be implemented in trunk and need not be backported.
      2. Using read-only mode allow oak-run from trunk to be safely connected to any of the old versions

      Attachments

        Issue Links

          There are no Sub-Tasks for this issue.

          Activity

            People

              chetanm Chetan Mehrotra
              chetanm Chetan Mehrotra
              Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: