Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-1544

Deadlock: IndexWriter.addIndexes(IndexReader[])

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.4
    • Fix Version/s: 2.4.1, 2.9
    • Component/s: core/index
    • Labels:
      None
    • Environment:

      should be ubiquitous... but, Win XP on a MacBookPro (Intel) under Eclipse (Ganymede) using default compiler/runtime

    • Lucene Fields:
      New

      Description

      A deadlock issue occurs under the following circumstances

      • IndexWriter.autoCommit == true
      • IndexWriter.directory contains multiple segments
      • IndexWriter.AddIndex(IndexReader[]) is invoked

      I put together a JUnit test that recreates the deadlock, which I've attached. It is the first test method, 'testAddIndexByIndexReader()'.

      In a nutshell, here is what happens:

      // 1) AddIndexes(IndexReader[]) acquires the write lock,
      // then begins optimization of destination index (this is
      // prior to adding any external segments).
      //
      // 2) Main thread starts a ConcurrentMergeScheduler.MergeThread
      // to merge the 2 segments.
      //
      // 3) Merging thread tries to acquire the read lock at
      // IndexWriter.blockAddIndexes(boolean) in
      // IndexWriter.StartCommit(), but cannot as...
      //
      // 4) Main thread still holds the write lock, and is
      // waiting for the IndexWriter.runningMerges data structure
      // to be devoid of merges with their optimize flag
      // set (IndexWriter.optimizeMergesPending()).

      1. LUCENE-1544.patch
        6 kB
        Michael McCandless
      2. TestAddIndexes.java
        6 kB
        Doug Sale

        Activity

        Hide
        dougsale Doug Sale added a comment -

        JUnit test.

        method/testcase that recreates deadlock: testAddIndexesByIndexReader()

        Show
        dougsale Doug Sale added a comment - JUnit test. method/testcase that recreates deadlock: testAddIndexesByIndexReader()
        Hide
        mikemccand Michael McCandless added a comment -

        Patch attached.

        I changed the up-front acquireWrite inside addIndexes(IndexReader[])
        to an acquireRead followed by upgradeReadToWrite.

        Once we remove autoCommit=true case we can greatly simplify IW – I'm
        looking forward to it!!

        I plan to commit in a day or so, and back port to 2.4.1.

        Show
        mikemccand Michael McCandless added a comment - Patch attached. I changed the up-front acquireWrite inside addIndexes(IndexReader[]) to an acquireRead followed by upgradeReadToWrite. Once we remove autoCommit=true case we can greatly simplify IW – I'm looking forward to it!! I plan to commit in a day or so, and back port to 2.4.1.
        Hide
        mikemccand Michael McCandless added a comment -

        Committed revision 746661 on trunk.

        Show
        mikemccand Michael McCandless added a comment - Committed revision 746661 on trunk.
        Hide
        mikemccand Michael McCandless added a comment -

        Committed revision 746662 on 2.4 branch.

        Thanks Doug!

        Show
        mikemccand Michael McCandless added a comment - Committed revision 746662 on 2.4 branch. Thanks Doug!

          People

          • Assignee:
            mikemccand Michael McCandless
            Reporter:
            dougsale Doug Sale
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development