Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-9662

CheckIndex should be concurrent

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • None
    • New

    Description

      I am watching a nightly benchmark run slowly run its CheckIndex step, using a single core out of the 128 cores the box has.

      It seems like this is an embarrassingly parallel problem, if the index has multiple segments, and would finish much more quickly on concurrent hardware if we did "thread per segment".

      If wanted to get even further concurrency, each part of the Lucene index that is checked is also independent, so it could be "thread per segment per part".

      Attachments

        Activity

          People

            Unassigned Unassigned
            mikemccand Michael McCandless
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 20h 50m
                20h 50m