Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-5800

When using NRTCachingDirectory, if the new created segment is in compound format, it is always created in cache(RAMDirectory). It will cause large segments referenced by IndexSearcher in memory.

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: core/store
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      When using NRTCachingDirectory, if the new created segment is in compound format, in the method of createOutput(String name,IOContext context), the real context is not used and uses IOContext.DEFAULT instead. So the estimatedMergeBytes or estimatedSegmentSize will always be smaller than maxMergeSizeBytes,maxCachedBytes, resulting in new created compound segment is always in cache(RAMDirectory). And these new large segments created by merging will be referenced by ReaderPool in the IndexWriter when using NRT feature, resulting in much memory used in process and actually these referenced large segments has already been sync to disk when commit.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              wangzj wangzj
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: