Lucene - Core
  1. Lucene - Core
  2. LUCENE-6094

IW.rollback can take forever when CMS has stalled threads

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.10.3, 5.0, 6.0
    • Component/s: core/index
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      CMS hard-stalls incoming threads for denial-of-service protection when merging cannot keep up with whatever is producing new segments.

      When you call IW.rollback, it asks all merges to abort, and a running merge will periodically check to see if it should abort.

      However, a stalled merge fails to check, which means rollback can take indefinitely long; I've seen this in Elasticsearch causing shutdown to take > 10 sec.

      1. LUCENE-6094.patch
        4 kB
        Michael McCandless

        Activity

        Hide
        Michael McCandless added a comment -

        Patch + test, I think it's ready. The test hangs w/o the fix ...

        Show
        Michael McCandless added a comment - Patch + test, I think it's ready. The test hangs w/o the fix ...
        Hide
        Robert Muir added a comment -

        +1

        Show
        Robert Muir added a comment - +1
        Hide
        ASF subversion and git services added a comment -

        Commit 1643508 from Michael McCandless in branch 'dev/trunk'
        [ https://svn.apache.org/r1643508 ]

        LUCENE-6094: allow IW.rollback to stop CMS's stalling too

        Show
        ASF subversion and git services added a comment - Commit 1643508 from Michael McCandless in branch 'dev/trunk' [ https://svn.apache.org/r1643508 ] LUCENE-6094 : allow IW.rollback to stop CMS's stalling too
        Hide
        ASF subversion and git services added a comment -

        Commit 1643509 from Michael McCandless in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1643509 ]

        LUCENE-6094: allow IW.rollback to stop CMS's stalling too

        Show
        ASF subversion and git services added a comment - Commit 1643509 from Michael McCandless in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1643509 ] LUCENE-6094 : allow IW.rollback to stop CMS's stalling too
        Hide
        Michael McCandless added a comment -

        Reopen for 4.10.3 backport

        Show
        Michael McCandless added a comment - Reopen for 4.10.3 backport
        Hide
        ASF subversion and git services added a comment -

        Commit 1643769 from Michael McCandless in branch 'dev/branches/lucene_solr_4_10'
        [ https://svn.apache.org/r1643769 ]

        LUCENE-6094: allow IW.rollback to stop CMS's stalling too

        Show
        ASF subversion and git services added a comment - Commit 1643769 from Michael McCandless in branch 'dev/branches/lucene_solr_4_10' [ https://svn.apache.org/r1643769 ] LUCENE-6094 : allow IW.rollback to stop CMS's stalling too
        Hide
        Anshum Gupta added a comment -

        Bulk close after 5.0 release.

        Show
        Anshum Gupta added a comment - Bulk close after 5.0 release.

          People

          • Assignee:
            Michael McCandless
            Reporter:
            Michael McCandless
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development