Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-5641

SimpleRateLimiter is too simple

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 4.8.1, 4.9, 6.0
    • None
    • None
    • New

    Description

      I was playing with merge throttling and discovered that our
      SimpleRateLimiter is throttling far more than requested; e.g. I asked
      for 50 MB/sec merge throttling, but it throttled at more like 8
      MB/sec.

      The problem is we are calling Thread.sleep on too-small (< 1 msec)
      times; on ordinary (non-real-time) JVMs, anything less than 1 msec is
      rounded up to 1 msec. Also, System.nanoTime() is somewhat costly.

      To fix this, I think we should aggregate the incoming byte count,
      until it crosses a threshold of enough bytes to warrant pausing.

      Attachments

        1. LUCENE-5641.patch
          9 kB
          Michael McCandless
        2. LUCENE-5641.patch
          8 kB
          Michael McCandless

        Activity

          People

            mikemccand Michael McCandless
            mikemccand Michael McCandless
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: