Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-8571

Don't block on FrozenBufferedUpdates#apply during IW#processEvents

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 7.5, 8.0
    • Fix Version/s: 7.7, 8.0
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      While indexing we try to apply frozen deletes packages concurrently
      on indexing threads if necessary. This is done in an opaque way via
      IndexWriter#processEvents. Yet, when we commit or refresh we have to
      ensure we apply all frozen update packages before we return.
      Today we execute the apply method in a blocking fashion which is unnecessary
      when we are in a IndexWriter#processEvents context, we block indexing
      threads while they could just continue since it's already being applied.
      We also might wait in BufferedUpdatesStream when we apply all necessary updates
      were we can continue with other work instead of waiting.
      This change also tries to apply the packages that are not currently applied
      first in order to not unnecessarily block.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                simonw Simon Willnauer
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1.5h
                  1.5h