Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-10216 Add concurrency to addIndexes(CodecReader…) API
  3. LUCENE-10476

Modify the findMerges policy for addIndexes(CodecReader[]) to create 1 segment per reader

Details

    • Sub-task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • None
    • New

    Description

      Once we have LUCENE-10216, we can trigger concurrent merges for addIndexes(CodecReader...) api, These merges are defined in the MergePolicy and triggered by the MergeScheduler.

      We can now create a merge policy that does a 1:1 addIndexes call, where the merger creates one segment each for every provided reader. It would create a faster, more concurrent addIndexes(CodecReader...), at the cost of deferring some merges to be done later in background.

      Which, I believe is similar to the behavior in addIndexes(Directory...) - all incoming segments are simply added to IW, and any merging happens in later in background.

      The change, would make both APIs behave consistently.

      Attachments

        Activity

          People

            Unassigned Unassigned
            vigyas Vigya Sharma
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: