Lucene - Core
  1. Lucene - Core
  2. LUCENE-2328

IndexWriter.synced field accumulates data leading to a Memory Leak

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 2.9.1, 2.9.2, 3.0, 3.0.1
    • Fix Version/s: 2.9.4, 3.0.3, 3.1, 4.0-ALPHA
    • Component/s: core/index
    • Labels:
      None
    • Environment:

      all

    • Lucene Fields:
      New

      Description

      I am running into a strange OutOfMemoryError. My small test application does
      index and delete some few files. This is repeated for 60k times. Optimization
      is run from every 2k times a file is indexed. Index size is 50KB. I did analyze
      the HeapDumpFile and realized that IndexWriter.synced field occupied more than
      half of the heap. That field is a private HashSet without a getter. Its task is
      to hold files which have been synced already.

      There are two calls to addAll and one call to add on synced but no remove or
      clear throughout the lifecycle of the IndexWriter instance.

      According to the Eclipse Memory Analyzer synced contains 32618 entries which
      look like file names "_e065_1.del" or "_e067.cfs"

      The index directory contains 10 files only.

      I guess synced is holding obsolete data

      1. LUCENE-2328.patch
        26 kB
        Earwin Burrfoot
      2. LUCENE-2328.patch
        28 kB
        Earwin Burrfoot
      3. LUCENE-2328.patch
        28 kB
        Earwin Burrfoot
      4. LUCENE-2328.patch
        28 kB
        Earwin Burrfoot
      5. LUCENE-2328.patch
        6 kB
        Michael McCandless

        Activity

        Mark Thomas made changes -
        Workflow Default workflow, editable Closed status [ 12563997 ] jira [ 12585470 ]
        Mark Thomas made changes -
        Workflow jira [ 12502126 ] Default workflow, editable Closed status [ 12563997 ]
        Uwe Schindler made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Michael McCandless made changes -
        Resolution Fixed [ 1 ]
        Status Reopened [ 4 ] Resolved [ 5 ]
        Michael McCandless made changes -
        Fix Version/s 2.9.4 [ 12315148 ]
        Fix Version/s 3.0.3 [ 12315147 ]
        Fix Version/s 3.1 [ 12314822 ]
        Michael McCandless made changes -
        Resolution Fixed [ 1 ]
        Status Resolved [ 5 ] Reopened [ 4 ]
        Michael McCandless made changes -
        Attachment LUCENE-2328.patch [ 12458462 ]
        Michael McCandless made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Earwin Burrfoot made changes -
        Attachment LUCENE-2328.patch [ 12439512 ]
        Earwin Burrfoot made changes -
        Attachment LUCENE-2328.patch [ 12439497 ]
        Earwin Burrfoot made changes -
        Attachment LUCENE-2328.patch [ 12439495 ]
        Michael McCandless made changes -
        Assignee Michael McCandless [ mikemccand ]
        Earwin Burrfoot made changes -
        Attachment LUCENE-2328.patch [ 12439449 ]
        Michael McCandless made changes -
        Field Original Value New Value
        Fix Version/s 3.1 [ 12314025 ]
        Gregor Kaczor created issue -

          People

          • Assignee:
            Michael McCandless
            Reporter:
            Gregor Kaczor
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - 1h
              1h
              Remaining:
              Remaining Estimate - 1h
              1h
              Logged:
              Time Spent - Not Specified
              Not Specified

                Development