Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-10031

Speedup to SortedDocIDMerger when sorting on low-cardinality fields

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • None
    • 8.10
    • None
    • None
    • New

    Description

      I've been looking at profiles of indexing with index sorting enabled and saw non-negligible time spent in SortedDocIDMerger. This isn't completely surprising as this little class is called on every document whenever merging postings, doc values, stored fields, etc.

      I'm especially interested in cases when the sort key is on a low cardinality field, so the priority queue doesn't get reordered often. I've been playing with a change to SortedDocIdMerger that makes merging significantly faster in that case.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              jpountz Adrien Grand
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h
                  1h