Lucene - Core
  1. Lucene - Core
  2. LUCENE-5374

Call processEvents before IndexWriter is closed

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 4.6
    • Fix Version/s: 4.6.1, 4.7, 6.0
    • Component/s: core/index
    • Labels:
      None
    • Lucene Fields:
      New, Patch Available

      Description

      We saw failures on jenkins that complain about processing events in the IW while the IW is already closed:

      com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=193, name=Thread-133, state=RUNNABLE, group=TGRP-TestIndexWriterWithThreads]
      Caused by: java.lang.RuntimeException: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
              at __randomizedtesting.SeedInfo.seed([3FAF37E1AFFB2502]:0)
              at org.apache.lucene.index.TestIndexWriterWithThreads$1.run(TestIndexWriterWithThreads.java:619)
      Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
              at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:645)
              at org.apache.lucene.index.IndexWriter.numDeletedDocs(IndexWriter.java:622)
              at org.apache.lucene.index.IndexWriter.segString(IndexWriter.java:4265)
              at org.apache.lucene.index.IndexWriter.publishFlushedSegment(IndexWriter.java:2324)
              at org.apache.lucene.index.DocumentsWriterFlushQueue$FlushTicket.publishFlushedSegment(DocumentsWriterFlushQueue.java:198)
              at org.apache.lucene.index.DocumentsWriterFlushQueue$FlushTicket.finishFlush(DocumentsWriterFlushQueue.java:213)
              at org.apache.lucene.index.DocumentsWriterFlushQueue$SegmentFlushTicket.publish(DocumentsWriterFlushQueue.java:249)
              at org.apache.lucene.index.DocumentsWriterFlushQueue.innerPurge(DocumentsWriterFlushQueue.java:116)
              at org.apache.lucene.index.DocumentsWriterFlushQueue.forcePurge(DocumentsWriterFlushQueue.java:138)
              at org.apache.lucene.index.DocumentsWriter.purgeBuffer(DocumentsWriter.java:185)
              at org.apache.lucene.index.IndexWriter.purge(IndexWriter.java:4634)
              at org.apache.lucene.index.DocumentsWriter$ForcedPurgeEvent.process(DocumentsWriter.java:701)
              at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4665)
              at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4657)
              at org.apache.lucene.index.IndexWriter.closeInternal(IndexWriter.java:1067)
              at org.apache.lucene.index.IndexWriter.rollbackInternal(IndexWriter.java:2106)
              at org.apache.lucene.index.IndexWriter.rollback(IndexWriter.java:2024)
              at org.apache.lucene.index.TestIndexWriterWithThreads$1.run(TestIndexWriterWithThreads.java:575)
      

      we need to process the events before we enter the finally block in IW#closeInternal

      1. LUCENE-5374.patch
        0.7 kB
        Simon Willnauer

        Activity

        Hide
        Simon Willnauer added a comment -

        here is a patch - I beasted this for a long time without a failure and without I was able to reproduce is quickly with this in a loop though:

         ant test  -Dtestcase=TestIndexWriterWithThreads -Dtests.method=testRollbackAndCommitWithThreads -Dtests.seed=3FAF37E1AFFB2502 -Dtests.slow=true -Dtests.locale=fr_FR -Dtests.timezone=Africa/Malabo -Dtests.file.encoding=UTF-8
        
        Show
        Simon Willnauer added a comment - here is a patch - I beasted this for a long time without a failure and without I was able to reproduce is quickly with this in a loop though: ant test -Dtestcase=TestIndexWriterWithThreads -Dtests.method=testRollbackAndCommitWithThreads -Dtests.seed=3FAF37E1AFFB2502 -Dtests.slow=true -Dtests.locale=fr_FR -Dtests.timezone=Africa/Malabo -Dtests.file.encoding=UTF-8
        Hide
        ASF subversion and git services added a comment -

        Commit 1552710 from Simon Willnauer in branch 'dev/trunk'
        [ https://svn.apache.org/r1552710 ]

        LUCENE-5374: Call IW#processEvents before IndexWriter is closed

        Show
        ASF subversion and git services added a comment - Commit 1552710 from Simon Willnauer in branch 'dev/trunk' [ https://svn.apache.org/r1552710 ] LUCENE-5374 : Call IW#processEvents before IndexWriter is closed
        Hide
        ASF subversion and git services added a comment -

        Commit 1552714 from Simon Willnauer in branch 'dev/branches/branch_4x'
        [ https://svn.apache.org/r1552714 ]

        LUCENE-5374: Call IW#processEvents before IndexWriter is closed

        Show
        ASF subversion and git services added a comment - Commit 1552714 from Simon Willnauer in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1552714 ] LUCENE-5374 : Call IW#processEvents before IndexWriter is closed
        Hide
        ASF subversion and git services added a comment -

        Commit 1556953 from Simon Willnauer in branch 'dev/branches/lucene_solr_4_6'
        [ https://svn.apache.org/r1556953 ]

        LUCENE-5374: Call IW#processEvents before IndexWriter is closed

        Show
        ASF subversion and git services added a comment - Commit 1556953 from Simon Willnauer in branch 'dev/branches/lucene_solr_4_6' [ https://svn.apache.org/r1556953 ] LUCENE-5374 : Call IW#processEvents before IndexWriter is closed

          People

          • Assignee:
            Simon Willnauer
            Reporter:
            Simon Willnauer
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development