Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-2328

IndexWriter.synced field accumulates data leading to a Memory Leak

Details

    • Bug
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 2.9.1, 2.9.2, 3.0, 3.0.1
    • 2.9.4, 3.0.3, 3.1, 4.0-ALPHA
    • core/index
    • None
    • all

    • New

    Description

      I am running into a strange OutOfMemoryError. My small test application does
      index and delete some few files. This is repeated for 60k times. Optimization
      is run from every 2k times a file is indexed. Index size is 50KB. I did analyze
      the HeapDumpFile and realized that IndexWriter.synced field occupied more than
      half of the heap. That field is a private HashSet without a getter. Its task is
      to hold files which have been synced already.

      There are two calls to addAll and one call to add on synced but no remove or
      clear throughout the lifecycle of the IndexWriter instance.

      According to the Eclipse Memory Analyzer synced contains 32618 entries which
      look like file names "_e065_1.del" or "_e067.cfs"

      The index directory contains 10 files only.

      I guess synced is holding obsolete data

      Attachments

        1. LUCENE-2328.patch
          6 kB
          Michael McCandless
        2. LUCENE-2328.patch
          28 kB
          Earwin Burrfoot
        3. LUCENE-2328.patch
          28 kB
          Earwin Burrfoot
        4. LUCENE-2328.patch
          28 kB
          Earwin Burrfoot
        5. LUCENE-2328.patch
          26 kB
          Earwin Burrfoot

        Activity

          People

            mikemccand Michael McCandless
            gregor.kaczor Gregor Kaczor
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - 1h
                1h
                Remaining:
                Remaining Estimate - 1h
                1h
                Logged:
                Time Spent - Not Specified
                Not Specified