Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-8590

Optimize DocValues update datastructures

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 7.7, 8.0
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      Today we are using a LinkedHashMap to buffer doc-values updates in
      BufferedUpdates. This on the one hand uses an Object based datastructure
      and on the other requires re-encoding the data into a more compact representation
      once the BufferedUpdates are frozen. This change uses a more compact represenation
      for the updates already in the BufferedUpdates in a parallel-array like datastructure
      that can be reused in FrozenBufferedDeletes. It also adds an much simpler to use
      API to consume the updates and allows for internal memory optimization for common
      case updates.

        Attachments

          Activity

            People

            • Assignee:
              simonw Simon Willnauer
              Reporter:
              simonw Simon Willnauer
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 4h 40m
                4h 40m