Lucene - Core
  1. Lucene - Core
  2. LUCENE-2255

IndexWriter.getReader() allocates file handles

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.2, 3.0.1, 4.0-ALPHA
    • Component/s: core/index
    • Labels:
      None
    • Environment:

      Ubuntu 9.10, Java 6

    • Lucene Fields:
      New

      Description

      I am not sure if this is a "bug" or really just me not reading the Javadocs right...

      The IR returned by IW.getReader() leaks file handles if you do not close() it, leading to starvation of the available file handles/process. If it was clear from the docs that this was a new reader and not some reference owned by the writer then this would probably be ok. But as I read the docs the reader is internally managed by the IW, which at first shot lead me to believe that I shouldn't close it.

      So perhaps the docs should be amended to clearly state that this is a caller-owns reader that must be closed? Attaching a simple app that illustrates the problem.

      1. LuceneManyCommits.java
        3 kB
        Mikkel Kamstrup Erlandsen

        Activity

        Hide
        Michael McCandless added a comment -

        This has already been fixed on trunk... we added this to the javadocs a while back:

        • <p>You must close the {@link IndexReader}

          returned by

        • this method once you are done using it.</p>

        While the returned reader used IndexWriter to determine which segments to open, the returned reader is fully independent of the writer. EG you can close the writer and continue using the reader. And you must close the reader.

        I'll backport that fix to 3.0 & 2.9.

        Show
        Michael McCandless added a comment - This has already been fixed on trunk... we added this to the javadocs a while back: <p>You must close the {@link IndexReader} returned by this method once you are done using it.</p> While the returned reader used IndexWriter to determine which segments to open, the returned reader is fully independent of the writer. EG you can close the writer and continue using the reader. And you must close the reader. I'll backport that fix to 3.0 & 2.9.
        Hide
        Jason Rutherglen added a comment -

        What keeps you from attaching a patch for the IW javadoc?

        Laziness and lack of motivation...

        I'll post something to the issue, how's that?

        Show
        Jason Rutherglen added a comment - What keeps you from attaching a patch for the IW javadoc? Laziness and lack of motivation... I'll post something to the issue, how's that?
        Hide
        Simon Willnauer added a comment -

        I see this coming up multiple times, we should document this properly in the javadoc and on the wiki. Jason, aren't you the NRT specialist here. What keeps you from attaching a patch for the IW javadoc?

        simon

        Show
        Simon Willnauer added a comment - I see this coming up multiple times, we should document this properly in the javadoc and on the wiki. Jason, aren't you the NRT specialist here. What keeps you from attaching a patch for the IW javadoc? simon
        Hide
        Jason Rutherglen added a comment -

        You're not the first to report this... I wonder if we should also do a weak ref type of dealie that auto closes the reader for users who aren't calling IR.close?

        To answer the question, yes, close on IR must be called. I can't directly do anything about the javadocs so I'll leave that for Mike.

        Show
        Jason Rutherglen added a comment - You're not the first to report this... I wonder if we should also do a weak ref type of dealie that auto closes the reader for users who aren't calling IR.close? To answer the question, yes, close on IR must be called. I can't directly do anything about the javadocs so I'll leave that for Mike.
        Hide
        Mikkel Kamstrup Erlandsen added a comment -

        Leaks file handles via IW.getReader()

        Show
        Mikkel Kamstrup Erlandsen added a comment - Leaks file handles via IW.getReader()

          People

          • Assignee:
            Unassigned
            Reporter:
            Mikkel Kamstrup Erlandsen
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development