Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-9539

Improve memory footprint of SortingCodecReader

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      SortingCodecReader is a very memory heavy since it needs to re-sort and load large parts of the index into memory. We can try to make it more efficient by using more compact internal data-structures, remove the caches it uses provided we define it's usage as a merge only reader wrapper. Ultimately we need to find a way to allow the reader or some other structure to minimize its heap memory. One way is to slice existing readers and merge them in multiple steps. There will be multiple steps towards a more useable version of this class.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              simonw Simon Willnauer
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 3h 20m
                3h 20m