Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-6063

Allow overriding ConcurrentMergeScheduler's denial-of-service protection

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.0, 6.0
    • Component/s: core/index
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      In LUCENE-5310 we explored improving CMS/SMS sharing/concurrency, but
      the issue never "converged", so I want to break out one small part of
      it here: the ability to override CMS's default "aggressive"
      denial-of-service protection where it forcefully stalls the incoming
      threads that are responsible for creating too many segments.

      More advanced applications can more gracefully handle the "too many
      merges" by e.g. slowing down the incoming indexing rate at a higher
      level.

      1. LUCENE-6063.patch
        5 kB
        Michael McCandless

        Activity

        Hide
        mikemccand Michael McCandless added a comment -

        Simple patch: I just moved the aggressive DOS protection into a new maybeStall() protected method.

        I also fixed newIWC to sometimes use a non-stalling CMS...

        Show
        mikemccand Michael McCandless added a comment - Simple patch: I just moved the aggressive DOS protection into a new maybeStall() protected method. I also fixed newIWC to sometimes use a non-stalling CMS...
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 1640456 from Michael McCandless in branch 'dev/trunk'
        [ https://svn.apache.org/r1640456 ]

        LUCENE-6063: allow overriding whether/how ConcurrentMergeScheduler stalls incoming threads when merges are falling behind

        Show
        jira-bot ASF subversion and git services added a comment - Commit 1640456 from Michael McCandless in branch 'dev/trunk' [ https://svn.apache.org/r1640456 ] LUCENE-6063 : allow overriding whether/how ConcurrentMergeScheduler stalls incoming threads when merges are falling behind
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 1640457 from Michael McCandless in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1640457 ]

        LUCENE-6063: allow overriding whether/how ConcurrentMergeScheduler stalls incoming threads when merges are falling behind

        Show
        jira-bot ASF subversion and git services added a comment - Commit 1640457 from Michael McCandless in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1640457 ] LUCENE-6063 : allow overriding whether/how ConcurrentMergeScheduler stalls incoming threads when merges are falling behind
        Hide
        anshumg Anshum Gupta added a comment -

        Bulk close after 5.0 release.

        Show
        anshumg Anshum Gupta added a comment - Bulk close after 5.0 release.

          People

          • Assignee:
            mikemccand Michael McCandless
            Reporter:
            mikemccand Michael McCandless
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development