Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-8571

Don't block on FrozenBufferedUpdates#apply during IW#processEvents

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 7.5, 8.0
    • 7.7, 8.0
    • None
    • None
    • New

    Description

      While indexing we try to apply frozen deletes packages concurrently
      on indexing threads if necessary. This is done in an opaque way via
      IndexWriter#processEvents. Yet, when we commit or refresh we have to
      ensure we apply all frozen update packages before we return.
      Today we execute the apply method in a blocking fashion which is unnecessary
      when we are in a IndexWriter#processEvents context, we block indexing
      threads while they could just continue since it's already being applied.
      We also might wait in BufferedUpdatesStream when we apply all necessary updates
      were we can continue with other work instead of waiting.
      This change also tries to apply the packages that are not currently applied
      first in order to not unnecessarily block.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              simonw Simon Willnauer
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1.5h
                  1.5h