Lucene - Core
  1. Lucene - Core
  2. LUCENE-1992

intermittent failure in TestIndexWriter. testExceptionDuringSync

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.1, 3.0
    • Component/s: core/index
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      common.test:
      
          [mkdir] Created dir: C:\Projects\lucene\trunk-full1\build\test
      
          [junit] Testsuite: org.apache.lucene.index.TestIndexWriter
      
          [junit] Tests run: 102, Failures: 0, Errors: 1, Time elapsed: 100,297sec
      
          [junit]
      
          [junit] Testcase: testExceptionDuringSync(org.apache.lucene.index.TestIndexWriter): Caused an ERROR
      
          [junit] _a.fnm
      
          [junit] java.io.FileNotFoundException: _a.fnm
      
          [junit]     at org.apache.lucene.store.MockRAMDirectory.openInput(MockRAMDirectory.java:226)
      
          [junit]     at org.apache.lucene.index.FieldInfos.<init>(FieldInfos.java:68)
      
          [junit]     at org.apache.lucene.index.SegmentReader$CoreReaders.<init>(SegmentReader.java:116)
      
          [junit]     at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:620)
      
          [junit]     at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:590)
      
          [junit]     at org.apache.lucene.index.DirectoryReader.<init>(DirectoryReader.java:104)
      
          [junit]     at org.apache.lucene.index.ReadOnlyDirectoryReader.<init>(ReadOnlyDirectoryReader.java:27)
      
          [junit]     at org.apache.lucene.index.DirectoryReader$1.doBody(DirectoryReader.java:74)
      
          [junit]     at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:704)
      
          [junit]     at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:69)
      
          [junit]     at org.apache.lucene.index.IndexReader.open(IndexReader.java:307)
      
          [junit]     at org.apache.lucene.index.IndexReader.open(IndexReader.java:193)
      
          [junit]     at org.apache.lucene.index.TestIndexWriter.testExceptionDuringSync(TestIndexWriter.java:2723)
      
          [junit]     at org.apache.lucene.util.LuceneTestCase.runBare(LuceneTestCase.java:206)
      
          [junit]
      
          [junit]
      
          [junit] Test org.apache.lucene.index.TestIndexWriter FAILED
      
      1. LUCENE-1992.patch
        2 kB
        Michael McCandless

        Activity

        Hide
        Michael McCandless added a comment -

        Patch attached. I'll commit shortly on trunk & 2.9.1.

        There is a thread hazard that happens when a merge is committing just
        as an exception is happening during sync. There merge actually
        changes the segmentInfos, but fails to checkpoint before exiting the
        sync block. If an exception during syncing is handled in another
        thread, this can then decRef the newly merged segment to 0, deleting
        it too early.

        Show
        Michael McCandless added a comment - Patch attached. I'll commit shortly on trunk & 2.9.1. There is a thread hazard that happens when a merge is committing just as an exception is happening during sync. There merge actually changes the segmentInfos, but fails to checkpoint before exiting the sync block. If an exception during syncing is handled in another thread, this can then decRef the newly merged segment to 0, deleting it too early.
        Hide
        Michael McCandless added a comment -

        Thanks for reporting this Uwe!

        Show
        Michael McCandless added a comment - Thanks for reporting this Uwe!
        Hide
        Uwe Schindler added a comment -

        From the patch:

        -    // We expect sync exceptions in the merge threads
        -    cms.setSuppressExceptions();
        

        Should this also applied to bw branch (2.4 for) 2.9 and (2.9 for) 3.0? I do not know what this call really does and what effect it has.

        Show
        Uwe Schindler added a comment - From the patch: - // We expect sync exceptions in the merge threads - cms.setSuppressExceptions(); Should this also applied to bw branch (2.4 for) 2.9 and (2.9 for) 3.0? I do not know what this call really does and what effect it has.
        Hide
        Michael McCandless added a comment -

        Should this also applied to bw branch (2.4 for) 2.9 and (2.9 for) 3.0?

        No, it can only be applied on trunk.

        That call tells ConcurrentMergeScheduler to expect exceptions during this test, which when autoCommit is true (which this test is doing everywhere except trunk) will happen because when a merge completes, it'll commit and call Directory.sync which throws the intentional exception.

        Show
        Michael McCandless added a comment - Should this also applied to bw branch (2.4 for) 2.9 and (2.9 for) 3.0? No, it can only be applied on trunk. That call tells ConcurrentMergeScheduler to expect exceptions during this test, which when autoCommit is true (which this test is doing everywhere except trunk) will happen because when a merge completes, it'll commit and call Directory.sync which throws the intentional exception.
        Hide
        Michael McCandless added a comment -

        Bulk close all 2.9.1 issues.

        Show
        Michael McCandless added a comment - Bulk close all 2.9.1 issues.

          People

          • Assignee:
            Michael McCandless
            Reporter:
            Michael McCandless
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development