Lucene - Core
  1. Lucene - Core
  2. LUCENE-5656

IndexWriter leaks CFS handles in some exceptional cases

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 4.8.1
    • Fix Version/s: 4.8.1, 4.9, 6.0
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      in trunk:

      ant test -Dtestcase=TestIndexWriterOutOfMemory -Dtests.method=testBasics -Dtests.seed=3D485DE153FCA22D -Dtests.nightly=true -Dtests.locale=no_NO -Dtests.timezone=CAT -Dtests.file.encoding=US-ASCII

      Seems to happen when an exception is thrown here:

         [junit4]   1> java.lang.OutOfMemoryError: Fake OutOfMemoryError
         [junit4]   1> 	at org.apache.lucene.index.TestIndexWriterOutOfMemory$2.eval(TestIndexWriterOutOfMemory.java:117)
         [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.maybeThrowDeterministicException(MockDirectoryWrapper.java:888)
         [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:575)
         [junit4]   1> 	at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:107)
         [junit4]   1> 	at org.apache.lucene.codecs.lucene45.Lucene45DocValuesProducer.<init>(Lucene45DocValuesProducer.java:84)
         [junit4]   1> 	at org.apache.lucene.codecs.lucene45.Lucene45DocValuesFormat.fieldsProducer(Lucene45DocValuesFormat.java:178)
         [junit4]   1> 	at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsReader.<init>(PerFieldDocValuesFormat.java:232)
         [junit4]   1> 	at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat.fieldsProducer(PerFieldDocValuesFormat.java:324)
         [junit4]   1> 	at org.apache.lucene.index.SegmentDocValues.newDocValuesProducer(SegmentDocValues.java:51)
         [junit4]   1> 	at org.apache.lucene.index.SegmentDocValues.getDocValuesProducer(SegmentDocValues.java:68)
         [junit4]   1> 	at org.apache.lucene.index.SegmentReader.initDocValuesProducers(SegmentReader.java:189)
         [junit4]   1> 	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:166)
         [junit4]   1> 	at org.apache.lucene.index.ReadersAndUpdates.writeFieldUpdates(ReadersAndUpdates.java:553)
         [junit4]   1> 	at org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:230)
         [junit4]   1> 	at org.apache.lucene.index.IndexWriter.applyAllDeletesAndUpdates(IndexWriter.java:3086)
         [junit4]   1> 	at org.apache.lucene.index.IndexWriter.maybeApplyDeletes(IndexWriter.java:3077)
         [junit4]   1> 	at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2791)
         [junit4]   1> 	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2940)
         [junit4]   1> 	at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2907)
      

      and the leak is from here:

         [junit4]    > Caused by: java.lang.RuntimeException: unclosed IndexInput: _0_Asserting_0.dvd
         [junit4]    > 	at org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:560)
         [junit4]    > 	at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:604)
         [junit4]    > 	at org.apache.lucene.codecs.lucene45.Lucene45DocValuesProducer.<init>(Lucene45DocValuesProducer.java:116)
         [junit4]    > 	at org.apache.lucene.codecs.lucene45.Lucene45DocValuesFormat.fieldsProducer(Lucene45DocValuesFormat.java:178)
         [junit4]    > 	at org.apache.lucene.codecs.asserting.AssertingDocValuesFormat.fieldsProducer(AssertingDocValuesFormat.java:61)
         [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsReader.<init>(PerFieldDocValuesFormat.java:232)
         [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat.fieldsProducer(PerFieldDocValuesFormat.java:324)
         [junit4]    > 	at org.apache.lucene.index.SegmentDocValues.newDocValuesProducer(SegmentDocValues.java:51)
         [junit4]    > 	at org.apache.lucene.index.SegmentDocValues.getDocValuesProducer(SegmentDocValues.java:68)
         [junit4]    > 	at org.apache.lucene.index.SegmentReader.initDocValuesProducers(SegmentReader.java:189)
         [junit4]    > 	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:116)
         [junit4]    > 	at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:133)
         [junit4]    > 	at org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:211)
         [junit4]    > 	at org.apache.lucene.index.IndexWriter.applyAllDeletesAndUpdates(IndexWriter.java:3086)
         [junit4]    > 	at org.apache.lucene.index.IndexWriter.maybeApplyDeletes(IndexWriter.java:3077)
         [junit4]    > 	at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2791)
         [junit4]    > 	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2940)
         [junit4]    > 	at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2907)
         [junit4]    > 	at org.apache.lucene.index.TestIndexWriterOutOfMemory.testBasics(TestIndexWriterOutOfMemory.java:209)
      

        Activity

        Hide
        ASF subversion and git services added a comment -

        Commit 1593238 from Robert Muir in branch 'dev/trunk'
        [ https://svn.apache.org/r1593238 ]

        LUCENE-5656: don't leak dv producers if one of them throws exception

        Show
        ASF subversion and git services added a comment - Commit 1593238 from Robert Muir in branch 'dev/trunk' [ https://svn.apache.org/r1593238 ] LUCENE-5656 : don't leak dv producers if one of them throws exception
        Hide
        ASF subversion and git services added a comment -

        Commit 1593239 from Robert Muir in branch 'dev/branches/branch_4x'
        [ https://svn.apache.org/r1593239 ]

        LUCENE-5656: don't leak dv producers if one of them throws exception

        Show
        ASF subversion and git services added a comment - Commit 1593239 from Robert Muir in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1593239 ] LUCENE-5656 : don't leak dv producers if one of them throws exception
        Hide
        ASF subversion and git services added a comment -

        Commit 1593241 from Robert Muir in branch 'dev/branches/lucene_solr_4_8'
        [ https://svn.apache.org/r1593241 ]

        LUCENE-5656: don't leak dv producers if one of them throws exception

        Show
        ASF subversion and git services added a comment - Commit 1593241 from Robert Muir in branch 'dev/branches/lucene_solr_4_8' [ https://svn.apache.org/r1593241 ] LUCENE-5656 : don't leak dv producers if one of them throws exception
        Hide
        Robert Muir added a comment -

        Shai spotted this easily, I committed!

        Show
        Robert Muir added a comment - Shai spotted this easily, I committed!
        Hide
        ASF subversion and git services added a comment -

        Commit 1593246 from Robert Muir in branch 'dev/branches/lucene_solr_4_8'
        [ https://svn.apache.org/r1593246 ]

        LUCENE-5656: add CHANGES

        Show
        ASF subversion and git services added a comment - Commit 1593246 from Robert Muir in branch 'dev/branches/lucene_solr_4_8' [ https://svn.apache.org/r1593246 ] LUCENE-5656 : add CHANGES

          People

          • Assignee:
            Unassigned
            Reporter:
            Robert Muir
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development