Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-2328

IndexWriter.synced field accumulates data leading to a Memory Leak

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.9.1, 2.9.2, 3.0, 3.0.1
    • Fix Version/s: 2.9.4, 3.0.3, 3.1, 4.0-ALPHA
    • Component/s: core/index
    • Labels:
      None
    • Environment:

      all

    • Lucene Fields:
      New

      Description

      I am running into a strange OutOfMemoryError. My small test application does
      index and delete some few files. This is repeated for 60k times. Optimization
      is run from every 2k times a file is indexed. Index size is 50KB. I did analyze
      the HeapDumpFile and realized that IndexWriter.synced field occupied more than
      half of the heap. That field is a private HashSet without a getter. Its task is
      to hold files which have been synced already.

      There are two calls to addAll and one call to add on synced but no remove or
      clear throughout the lifecycle of the IndexWriter instance.

      According to the Eclipse Memory Analyzer synced contains 32618 entries which
      look like file names "_e065_1.del" or "_e067.cfs"

      The index directory contains 10 files only.

      I guess synced is holding obsolete data

        Attachments

        1. LUCENE-2328.patch
          26 kB
          Earwin Burrfoot
        2. LUCENE-2328.patch
          28 kB
          Earwin Burrfoot
        3. LUCENE-2328.patch
          28 kB
          Earwin Burrfoot
        4. LUCENE-2328.patch
          28 kB
          Earwin Burrfoot
        5. LUCENE-2328.patch
          6 kB
          Michael McCandless

          Activity

            People

            • Assignee:
              mikemccand Michael McCandless
              Reporter:
              gregor.kaczor Gregor Kaczor
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 1h
                1h
                Remaining:
                Remaining Estimate - 1h
                1h
                Logged:
                Time Spent - Not Specified
                Not Specified