Uploaded image for project: 'Jackrabbit Oak'
  1. Jackrabbit Oak
  2. OAK-2247

CopyOnWriteDirectory implementation for Lucene for use in indexing

    XMLWordPrintableJSON

    Details

      Description

      Currently a Lucene index when is written directly to OakDirectory. For reindex case it might happen that Lucene merge policy read the written index files again and then perform a sgement merge. This might have lower performance when OakDirectroy is writing to remote storage.

      Instead of that we can implement a CopyOnWriteDirectory on similar lines to OAK-1724 where CopyOnReadDirectory support copies the index locally for faster access.

      At high level flow would be

      1. While writing index the index file is first written to local directory
      2. Any write is done locally and once a file is written its written asynchronously to OakDirectory
      3. When IndexWriter is closed it would wait untill all the write is completed

      This needs to be benchmarked with existing reindex timings to see it its actually beneficial

        Attachments

        1. OAK-2247-v1.patch
          79 kB
          Chetan Mehrotra

          Issue Links

            Activity

              People

              • Assignee:
                chetanm Chetan Mehrotra
                Reporter:
                chetanm Chetan Mehrotra
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: