Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-9536

Optimize OrdinalMap when one segment contains all distinct values?

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 8.8
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      For doc values that are not too high cardinality, it seems common to have some large segments that contain all distinct values (plus many small segments who are missing some values). In this case, we could check if the first segment ords map perfectly to global ords and if so store `globalOrdDeltas` and `firstSegments` as `LongValues.ZEROES`. This could save a small amount of space.

      I don’t think it would help a huge amount, especially since the optimization might only kick in with small/ medium cardinalities, which don’t create huge `OrdinalMap` instances anyways? But it is simple and seemed worth mentioning.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              jtibshirani Julie Tibshirani
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 3h 40m
                3h 40m