Lucene - Core
  1. Lucene - Core
  2. LUCENE-6063

Allow overriding ConcurrentMergeScheduler's denial-of-service protection

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.0, 6.0
    • Component/s: core/index
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      In LUCENE-5310 we explored improving CMS/SMS sharing/concurrency, but
      the issue never "converged", so I want to break out one small part of
      it here: the ability to override CMS's default "aggressive"
      denial-of-service protection where it forcefully stalls the incoming
      threads that are responsible for creating too many segments.

      More advanced applications can more gracefully handle the "too many
      merges" by e.g. slowing down the incoming indexing rate at a higher
      level.

      1. LUCENE-6063.patch
        5 kB
        Michael McCandless

        Activity

        Hide
        Michael McCandless added a comment -

        Simple patch: I just moved the aggressive DOS protection into a new maybeStall() protected method.

        I also fixed newIWC to sometimes use a non-stalling CMS...

        Show
        Michael McCandless added a comment - Simple patch: I just moved the aggressive DOS protection into a new maybeStall() protected method. I also fixed newIWC to sometimes use a non-stalling CMS...
        Hide
        ASF subversion and git services added a comment -

        Commit 1640456 from Michael McCandless in branch 'dev/trunk'
        [ https://svn.apache.org/r1640456 ]

        LUCENE-6063: allow overriding whether/how ConcurrentMergeScheduler stalls incoming threads when merges are falling behind

        Show
        ASF subversion and git services added a comment - Commit 1640456 from Michael McCandless in branch 'dev/trunk' [ https://svn.apache.org/r1640456 ] LUCENE-6063 : allow overriding whether/how ConcurrentMergeScheduler stalls incoming threads when merges are falling behind
        Hide
        ASF subversion and git services added a comment -

        Commit 1640457 from Michael McCandless in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1640457 ]

        LUCENE-6063: allow overriding whether/how ConcurrentMergeScheduler stalls incoming threads when merges are falling behind

        Show
        ASF subversion and git services added a comment - Commit 1640457 from Michael McCandless in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1640457 ] LUCENE-6063 : allow overriding whether/how ConcurrentMergeScheduler stalls incoming threads when merges are falling behind
        Hide
        Anshum Gupta added a comment -

        Bulk close after 5.0 release.

        Show
        Anshum Gupta added a comment - Bulk close after 5.0 release.

          People

          • Assignee:
            Michael McCandless
            Reporter:
            Michael McCandless
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development