Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-7700

Move throughput control and merge aborting out of IndexWriter's core?


    • Improvement
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • None
    • 6.5, 7.0
    • None
    • None
    • New


      Here is a bit of a background:

      • I wanted to implement a custom merging strategy that would have a custom i/o flow control (global),
      • currently, the CMS is tightly coupled with a few classes – MergeRateLimiter, OneMerge, IndexWriter.

      Looking at the code it seems to me that everything with respect to I/O control could be nicely pulled out into classes that explicitly control the merging process, that is only MergePolicy and MergeScheduler. By default, one could even run without any additional I/O accounting overhead (which is currently in there, even if one doesn't use the CMS's throughput control).

      Such refactoring would also give a chance to nicely move things where they belong – job aborting into OneMerge (currently in RateLimiter), rate limiter lifecycle bound to OneMerge (MergeScheduler could then use per-merge or global accounting, as it pleases).

      Just a thought and some initial refactorings for discussion.


        1. LUCENE-7700.patch
          16 kB
          Dawid Weiss
        2. LUCENE-7700.patch
          39 kB
          Dawid Weiss
        3. LUCENE-7700.patch
          42 kB
          Dawid Weiss
        4. LUCENE-7700.patch
          43 kB
          Dawid Weiss



            dweiss Dawid Weiss
            dweiss Dawid Weiss
            0 Vote for this issue
            3 Start watching this issue