Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-7700

Move throughput control and merge aborting out of IndexWriter's core?

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • None
    • 6.5, 7.0
    • None
    • None
    • New

    Description

      Here is a bit of a background:

      • I wanted to implement a custom merging strategy that would have a custom i/o flow control (global),
      • currently, the CMS is tightly coupled with a few classes – MergeRateLimiter, OneMerge, IndexWriter.

      Looking at the code it seems to me that everything with respect to I/O control could be nicely pulled out into classes that explicitly control the merging process, that is only MergePolicy and MergeScheduler. By default, one could even run without any additional I/O accounting overhead (which is currently in there, even if one doesn't use the CMS's throughput control).

      Such refactoring would also give a chance to nicely move things where they belong – job aborting into OneMerge (currently in RateLimiter), rate limiter lifecycle bound to OneMerge (MergeScheduler could then use per-merge or global accounting, as it pleases).

      Just a thought and some initial refactorings for discussion.

      Attachments

        1. LUCENE-7700.patch
          43 kB
          Dawid Weiss
        2. LUCENE-7700.patch
          42 kB
          Dawid Weiss
        3. LUCENE-7700.patch
          39 kB
          Dawid Weiss
        4. LUCENE-7700.patch
          16 kB
          Dawid Weiss

        Activity

          People

            dweiss Dawid Weiss
            dweiss Dawid Weiss
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: