Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-6119

Add auto-io-throttle to ConcurrentMergeScheduler

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.0, 6.0
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      This method returns number of "incoming" bytes IW has written since it
      was opened, excluding merging.

      It tracks flushed segments, new commits (segments_N), incoming
      files/segments by addIndexes, newly written live docs / doc values
      updates files.

      It's an easy statistic for IW to track and should be useful to help
      applications more intelligently set defaults for IO throttling
      (RateLimiter).

      For example, an application that does hardly any indexing but finally
      triggered a large merge can afford to heavily throttle that large
      merge so it won't interfere with ongoing searches.

      But an application that's causing IW to write new bytes at 50 MB/sec
      must set a correspondingly higher IO throttling otherwise merges will
      clearly fall behind.

        Attachments

        1. LUCENE-6119.patch
          143 kB
          Michael McCandless
        2. LUCENE-6119.patch
          143 kB
          Michael McCandless
        3. LUCENE-6119.patch
          124 kB
          Michael McCandless
        4. LUCENE-6119.patch
          66 kB
          Michael McCandless
        5. LUCENE-6119.patch
          35 kB
          Michael McCandless
        6. LUCENE-6119.patch
          12 kB
          Michael McCandless

          Activity

            People

            • Assignee:
              mikemccand Michael McCandless
              Reporter:
              mikemccand Michael McCandless
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: