Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-2387

IndexWriter retains references to Readers used in Fields (memory leak)

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 3.0.1
    • 2.9.3, 3.0.2, 3.1, 4.0-ALPHA
    • None
    • None
    • New

    Description

      As described in [1] IndexWriter retains references to Reader used in Fields and that can lead to big memory leaks when using tika's ParsingReaders (as those can take 1MB per ParsingReader).

      [2] shows a screenshot of the reference chain to the Reader from the IndexWriter taken with Eclipse MAT (Memory Analysis Tool) . The chain is the following:

      IndexWriter -> DocumentsWriter -> DocumentsWriterThreadState -> DocFieldProcessorPerThread -> DocFieldProcessorPerField -> Fieldable -> Field (fieldsData)

      -------------
      [1] http://markmail.org/thread/ndmcgffg2mnwjo47
      [2] http://skitch.com/ecerulm/n7643/eclipse-memory-analyzer

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            mikemccand Michael McCandless
            ecerulm Ruben Laguna
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment