Lucene - Core
  1. Lucene - Core
  2. LUCENE-6579

Unexpected merge exceptions should be tragic to IndexWriter

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.3, 6.0
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      Today our behavior is weird: we will fail the merge (which is running in a background thread if you are using the default CMS), pause for 1.0 seconds, and then the next chance we get, kick off the merge again.

      I think this is a poor default, e.g. on disk full we will just keep "trying" and filling up disk again, wasting IO/CPU.

      I think IW should declare this a tragedy instead?

      1. LUCENE-6579.patch
        27 kB
        Michael McCandless

        Issue Links

          Activity

          Hide
          Robert Muir added a comment -

          +1

          Show
          Robert Muir added a comment - +1
          Hide
          Michael McCandless added a comment -

          Here's a tentative patch ... it was tricky because calling IW.tragicEvent from a merge thread quickly lead to deadlock. I had to relax a few locking/blocking places to work around that ...

          I also had to fix some tests to expected that IW is closed after hitting a merge exc, but I'm sure there's a long tail still there ... this will be destabilizing at first.

          I still need to add a dedicated test that confirms IW is closed after a merge exc.

          Show
          Michael McCandless added a comment - Here's a tentative patch ... it was tricky because calling IW.tragicEvent from a merge thread quickly lead to deadlock. I had to relax a few locking/blocking places to work around that ... I also had to fix some tests to expected that IW is closed after hitting a merge exc, but I'm sure there's a long tail still there ... this will be destabilizing at first. I still need to add a dedicated test that confirms IW is closed after a merge exc.
          Hide
          ASF subversion and git services added a comment -

          Commit 1690886 from Michael McCandless in branch 'dev/trunk'
          [ https://svn.apache.org/r1690886 ]

          LUCENE-6579: exceptions during merging are now tragic

          Show
          ASF subversion and git services added a comment - Commit 1690886 from Michael McCandless in branch 'dev/trunk' [ https://svn.apache.org/r1690886 ] LUCENE-6579 : exceptions during merging are now tragic
          Hide
          ASF subversion and git services added a comment -

          Commit 1690887 from Michael McCandless in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1690887 ]

          LUCENE-6579: exceptions during merging are now tragic

          Show
          ASF subversion and git services added a comment - Commit 1690887 from Michael McCandless in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1690887 ] LUCENE-6579 : exceptions during merging are now tragic
          Hide
          ASF subversion and git services added a comment -

          Commit 1690918 from Michael McCandless in branch 'dev/trunk'
          [ https://svn.apache.org/r1690918 ]

          LUCENE-6579: fix this test case to cope with tragedy

          Show
          ASF subversion and git services added a comment - Commit 1690918 from Michael McCandless in branch 'dev/trunk' [ https://svn.apache.org/r1690918 ] LUCENE-6579 : fix this test case to cope with tragedy
          Hide
          ASF subversion and git services added a comment -

          Commit 1690919 from Michael McCandless in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1690919 ]

          LUCENE-6579: fix this test case to cope with tragedy

          Show
          ASF subversion and git services added a comment - Commit 1690919 from Michael McCandless in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1690919 ] LUCENE-6579 : fix this test case to cope with tragedy
          Hide
          Shalin Shekhar Mangar added a comment -

          Bulk close for 5.3.0 release

          Show
          Shalin Shekhar Mangar added a comment - Bulk close for 5.3.0 release

            People

            • Assignee:
              Michael McCandless
              Reporter:
              Michael McCandless
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development