Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: IOContext branch
    • Component/s: core/store
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      Large merges can mess up searches and increase NRT reopen time (see
      http://blog.mikemccandless.com/2011/06/lucenes-near-real-time-search-is-fast.html).

      A simple rate limiter improves the spikey NRT reopen times during big
      merges, so I think we should somehow make this possible. Likely this
      would reduce impact on searches as well.

      Typically apps that do indexing and searching on same box are in no
      rush to see the merges complete so this is a good tradeoff.

      1. LUCENE-3203.patch
        10 kB
        Michael McCandless
      2. LUCENE-3203.patch
        7 kB
        Michael McCandless

        Issue Links

          Activity

          Hide
          Michael McCandless added a comment -

          Patch, with a hacked up a prototype impl, but I don't think we should
          commit it like this. Instead, I think we should wait for IOContext,
          and then Dir impls can allow app to specify max merge write rate.

          Show
          Michael McCandless added a comment - Patch, with a hacked up a prototype impl, but I don't think we should commit it like this. Instead, I think we should wait for IOContext, and then Dir impls can allow app to specify max merge write rate.
          Hide
          Michael McCandless added a comment -

          New patch, applies to the IOContext branch. I think it's committable!

          It adds set/getMaxMergeWriteMBPerSec methods to FSDirectory.

          Show
          Michael McCandless added a comment - New patch, applies to the IOContext branch. I think it's committable! It adds set/getMaxMergeWriteMBPerSec methods to FSDirectory.

            People

            • Assignee:
              Michael McCandless
              Reporter:
              Michael McCandless
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development