Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-7141

OfflineSorter shouldn't always forceMerge in the end

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      Today it always does a final merge, to collapse all segments into a single segment.

      But typically the caller is going to re-iterate all values anyway, to go off and build an FST or a BKD tree or something, and so that final forceMerge is often not necessary and the merging can be done on the fly when the caller consumes the result.

      This is somewhat tricky to do ... I'd like to break it into steps, starting with fixing the ByteSequencesReader API to implement BytesRefIterator instead of its own read(BytesRefBuilder) method as a first step.

        Attachments

        1. LUCENE-7141.patch
          16 kB
          Michael McCandless

          Activity

            People

            • Assignee:
              mikemccand Michael McCandless
              Reporter:
              mikemccand Michael McCandless
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: