Lucene - Core
  1. Lucene - Core
  2. LUCENE-4071

DWStallControl can deadlock IW if no flushes are running / pending

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 4.0-ALPHA
    • Fix Version/s: 4.0-ALPHA
    • Component/s: core/index
    • Labels:
      None
    • Lucene Fields:
      New, Patch Available

      Description

      DWStallControl currently only checks if the net bytes used by all the DWPT and deletes exceeds the stall limit (2*MAX_RAM_BUFFER). This is generally a very good default but in certain situations we can exceed this limit even without an ongoing flush. Stalling is used to prevent IW overloading due to slow flushes etc. which should not happen too often in practice. With a smallish RAM Buffer and a bigger document we can easily get into the stage where we stall the DW without a chance to free up the memory.

      I think we should make sure that a pending or running flush can free up enough memory to unstall.

      1. LUCENE-4071.patch
        7 kB
        Simon Willnauer
      2. LUCENE-4071.patch
        5 kB
        Simon Willnauer

        Activity

        Simon Willnauer created issue -
        Hide
        Simon Willnauer added a comment -

        here is a patch

        Show
        Simon Willnauer added a comment - here is a patch
        Simon Willnauer made changes -
        Field Original Value New Value
        Attachment LUCENE-4071.patch [ 12528402 ]
        Hide
        Michael McCandless added a comment -

        Nice catch!

        Patch looks good.

        Show
        Michael McCandless added a comment - Nice catch! Patch looks good.
        Hide
        Uwe Schindler added a comment -

        Thanks to SDDS-Jenkins

        Show
        Uwe Schindler added a comment - Thanks to SDDS-Jenkins
        Hide
        Simon Willnauer added a comment -

        updated patch... I will commit soon

        Show
        Simon Willnauer added a comment - updated patch... I will commit soon
        Simon Willnauer made changes -
        Attachment LUCENE-4071.patch [ 12528477 ]
        Hide
        Simon Willnauer added a comment -

        committed to trunk

        Show
        Simon Willnauer added a comment - committed to trunk
        Simon Willnauer made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Uwe Schindler made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Resolved Resolved
        23h 1m 1 Simon Willnauer 22/May/12 08:09
        Resolved Resolved Closed Closed
        353d 2h 33m 1 Uwe Schindler 10/May/13 10:42

          People

          • Assignee:
            Simon Willnauer
            Reporter:
            Simon Willnauer
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development