Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-6119

Add auto-io-throttle to ConcurrentMergeScheduler

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 5.0, 6.0
    • None
    • None
    • New

    Description

      This method returns number of "incoming" bytes IW has written since it
      was opened, excluding merging.

      It tracks flushed segments, new commits (segments_N), incoming
      files/segments by addIndexes, newly written live docs / doc values
      updates files.

      It's an easy statistic for IW to track and should be useful to help
      applications more intelligently set defaults for IO throttling
      (RateLimiter).

      For example, an application that does hardly any indexing but finally
      triggered a large merge can afford to heavily throttle that large
      merge so it won't interfere with ongoing searches.

      But an application that's causing IW to write new bytes at 50 MB/sec
      must set a correspondingly higher IO throttling otherwise merges will
      clearly fall behind.

      Attachments

        1. LUCENE-6119.patch
          143 kB
          Michael McCandless
        2. LUCENE-6119.patch
          143 kB
          Michael McCandless
        3. LUCENE-6119.patch
          124 kB
          Michael McCandless
        4. LUCENE-6119.patch
          66 kB
          Michael McCandless
        5. LUCENE-6119.patch
          35 kB
          Michael McCandless
        6. LUCENE-6119.patch
          12 kB
          Michael McCandless

        Issue Links

          Activity

            People

              mikemccand Michael McCandless
              mikemccand Michael McCandless
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Slack

                  Issue deployment