Lucene - Core
  1. Lucene - Core
  2. LUCENE-1397

When BG merge hits an exception, optimize sometimes throws an IOException missing the root cause

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 2.4
    • Fix Version/s: 2.4, 2.9
    • Component/s: core/index
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      When IndexWriter.optimize() is called, ConcurrentMergeScheduler will
      run the requested merges with background threads and optimize() will
      wait for these merges to complete.

      If a merge hits an exception, it records the root cause exception such
      that optimize can then retrieve this root cause and throw its own
      exception, with the root cause.

      But there is a bug: sometimes, the fact that an exception occurred on
      a merge is recorded, but the root cause is missing. In this cause,
      optimize() still throws an exception (correctly indicating that the
      optimize() has not finished successfully), but it's not helpful
      because it's missing the root cause. You must then go find the root
      cause in the JRE's stderr logs.

      This has hit a few users on this lists, most recently:

      http://www.nabble.com/Background-merge-hit-exception-td19540409.html#a19540409

      I found the isssue, and finally got a unit test to intermittently show
      it. It's a simple thread safety issue: in a finally clause in
      IndexWriter.merge we record the fact that the merge hit an exception
      before actually setting the root cause, and then only in
      ConcurrentMergeScheduler's exception handler do we set the root
      cause. If the optimize thread is scheduled in between these two, it
      can throw an exception missing its root cause.

      The fix is straightforward. I plan to commit to 2.4 & 2.9.

      1. LUCENE-1397.patch
        5 kB
        Michael McCandless

        Activity

        Hide
        Michael McCandless added a comment -

        Attached patch. I plan to commit in a day or so, and then roll another RC.

        Show
        Michael McCandless added a comment - Attached patch. I plan to commit in a day or so, and then roll another RC.
        Hide
        Michael McCandless added a comment -

        Committed revision 698025 (trunk) and 698026 (2.4)

        Show
        Michael McCandless added a comment - Committed revision 698025 (trunk) and 698026 (2.4)

          People

          • Assignee:
            Michael McCandless
            Reporter:
            Michael McCandless
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development