Lucene - Core
  1. Lucene - Core
  2. LUCENE-4638

If IndexWriter is interrupted on close and is using a channel (mmap/nio), it can throw a ClosedByInterruptException and prevent you from opening a new IndexWriter in the same proceses if you are using Native locks.

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 4.9, 5.0
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      The ClosedByInterruptException will prevent the index from being unlocked in close. If you try and close again, the call will hang. If you are using native locks and try to open a new IndexWriter, it will fail to get the lock. If you try IW#forceUnlock, it wont work because the not fully closed IW will still have the lock.

      ideas:

      • On ClosedByInterruptException, IW should continue trying to close what it can and unlock the index? Generally I have see the exception trigger in commitInternal.
      • We should add a non static forceUnlock to IW that lets you remove the lock and start a new IW?
      • We should make the lock protected so IW sub classes could unlock the index in advanced use cases?
      • others?

        Issue Links

          Activity

          Hide
          Michael McCandless added a comment -

          Hmm why does the 2nd call to close hang? Do you have the original exc?

          IW.rollback() should do a "better job" closing and releasing the lock, and in general on getting an exception from IW.close I think it's the only real recourse you have (ie, it's hard to know what docs you lost due to that exception).

          Also, I think LUCENE-4246 (IW.close should "just close", not wait for merges, commit, etc.) would improve this situation because then close would reliably release the lock.

          Show
          Michael McCandless added a comment - Hmm why does the 2nd call to close hang? Do you have the original exc? IW.rollback() should do a "better job" closing and releasing the lock, and in general on getting an exception from IW.close I think it's the only real recourse you have (ie, it's hard to know what docs you lost due to that exception). Also, I think LUCENE-4246 (IW.close should "just close", not wait for merges, commit, etc.) would improve this situation because then close would reliably release the lock.
          Hide
          Mark Miller added a comment -

          I should be able to find what it was hanging on, but a lot of logs to look back through. I can probably reproduce more easily instead when I get home tonight. If I remember right, it was trying to open an mmap input or something and if I remember right, it was just blocking. I'll reproduce and report the exact details.

          Show
          Mark Miller added a comment - I should be able to find what it was hanging on, but a lot of logs to look back through. I can probably reproduce more easily instead when I get home tonight. If I remember right, it was trying to open an mmap input or something and if I remember right, it was just blocking. I'll reproduce and report the exact details.
          Hide
          Mark Miller added a comment -

          I have not had a chance to duplicate the hang yet - using fullmetaljenkins to work on some other bugs. I really could use a resolution to this though.

          Currently, the advice for cleaning up after an IndexWriter in the javadoc is broken with native locks. You can't necessarily call close twice and you can't unlock using the static unlock method.

          Here is a patch that provides a way for users to use the unlock in a finally pattern that is safe for native locks.

          It adds a forceUnlock method to IndexWriter that is not static.

          Show
          Mark Miller added a comment - I have not had a chance to duplicate the hang yet - using fullmetaljenkins to work on some other bugs. I really could use a resolution to this though. Currently, the advice for cleaning up after an IndexWriter in the javadoc is broken with native locks. You can't necessarily call close twice and you can't unlock using the static unlock method. Here is a patch that provides a way for users to use the unlock in a finally pattern that is safe for native locks. It adds a forceUnlock method to IndexWriter that is not static.
          Hide
          Mark Miller added a comment -

          Probably want to put a != null check around the writeLock.

          Show
          Mark Miller added a comment - Probably want to put a != null check around the writeLock.
          Hide
          Michael McCandless added a comment -

          I don't think we should rush a fix here.

          Let's see if rollback would have fixed it (and really javadocs should state that as the "recovery" if you hit exc during close), and let's understand what was hanging in the 2nd call to IW.close.

          I think a new forceUnluck method in IndexWriter is too dangerous because the IndexWriter technically is still open so the app can continue to do ops after releasing the lock.

          Show
          Michael McCandless added a comment - I don't think we should rush a fix here. Let's see if rollback would have fixed it (and really javadocs should state that as the "recovery" if you hit exc during close), and let's understand what was hanging in the 2nd call to IW.close. I think a new forceUnluck method in IndexWriter is too dangerous because the IndexWriter technically is still open so the app can continue to do ops after releasing the lock.
          Hide
          Mark Miller added a comment -

          I think a new forceUnluck method in IndexWriter is too dangerous

          It's the same as the current static unlock method and javadocd the same.

          I'm okay with it not being in Lucene though - I figure users would like to avoid this bug as well, but simply making the lock factory protected exposes it in an advanced enough way that it couldnt be considered dangerous. That would let me gid rid of this bug as well.

          Let's see if rollback would have fixed it

          I'll try that.

          Show
          Mark Miller added a comment - I think a new forceUnluck method in IndexWriter is too dangerous It's the same as the current static unlock method and javadocd the same. I'm okay with it not being in Lucene though - I figure users would like to avoid this bug as well, but simply making the lock factory protected exposes it in an advanced enough way that it couldnt be considered dangerous. That would let me gid rid of this bug as well. Let's see if rollback would have fixed it I'll try that.
          Hide
          Michael McCandless added a comment -

          simply making the lock factory protected exposes it in an advanced enough way that it couldnt be considered dangerous. That would let me gid rid of this bug as well.

          I think that's a good solution for this issue?

          It would still be nice to know if rollback resolves it (it's supposed to!), and why the 2nd IW.close() hangs (which is weird).

          Show
          Michael McCandless added a comment - simply making the lock factory protected exposes it in an advanced enough way that it couldnt be considered dangerous. That would let me gid rid of this bug as well. I think that's a good solution for this issue? It would still be nice to know if rollback resolves it (it's supposed to!), and why the 2nd IW.close() hangs (which is weird ).
          Hide
          Robert Muir added a comment -

          if thats what gets committed, please keep the issue open in that case.

          This kind of behavior in close is outright buggy. its because its doing too much.

          Show
          Robert Muir added a comment - if thats what gets committed, please keep the issue open in that case. This kind of behavior in close is outright buggy. its because its doing too much.
          Hide
          Robert Muir added a comment -

          Also i would view it as a temporary solution, like until we have time to unfuck close() to not do so much.

          I dont care that the issue is controversial. Its time to bring this to a head. I'm good at that.

          Show
          Robert Muir added a comment - Also i would view it as a temporary solution, like until we have time to unfuck close() to not do so much. I dont care that the issue is controversial. Its time to bring this to a head. I'm good at that.
          Hide
          Robert Muir added a comment -

          just a start so there is no wimpy solution committed permanently because close() does too much. I dont want 4.1 released with that solution.

          Show
          Robert Muir added a comment - just a start so there is no wimpy solution committed permanently because close() does too much. I dont want 4.1 released with that solution.
          Hide
          Commit Tag Bot added a comment -

          [trunk commit] Mark Robert Miller
          http://svn.apache.org/viewvc?view=revision&revision=1425561

          LUCENE-4638, SOLR-3180: try using the IW's writeLock to unlock

          Show
          Commit Tag Bot added a comment - [trunk commit] Mark Robert Miller http://svn.apache.org/viewvc?view=revision&revision=1425561 LUCENE-4638 , SOLR-3180 : try using the IW's writeLock to unlock
          Hide
          Commit Tag Bot added a comment -

          [branch_4x commit] Mark Robert Miller
          http://svn.apache.org/viewvc?view=revision&revision=1425563

          LUCENE-4638, SOLR-3180: try using the IW's writeLock to unlock

          Show
          Commit Tag Bot added a comment - [branch_4x commit] Mark Robert Miller http://svn.apache.org/viewvc?view=revision&revision=1425563 LUCENE-4638 , SOLR-3180 : try using the IW's writeLock to unlock
          Hide
          Commit Tag Bot added a comment -

          [trunk commit] Mark Robert Miller
          http://svn.apache.org/viewvc?view=revision&revision=1425574

          LUCENE-4638, SOLR-3180: revert for now (try using the IW's writeLock to unlock)

          Show
          Commit Tag Bot added a comment - [trunk commit] Mark Robert Miller http://svn.apache.org/viewvc?view=revision&revision=1425574 LUCENE-4638 , SOLR-3180 : revert for now (try using the IW's writeLock to unlock)
          Hide
          Commit Tag Bot added a comment -

          [branch_4x commit] Mark Robert Miller
          http://svn.apache.org/viewvc?view=revision&revision=1425576

          LUCENE-4638, SOLR-3180: revert for now (try using the IW's writeLock to unlock)

          Show
          Commit Tag Bot added a comment - [branch_4x commit] Mark Robert Miller http://svn.apache.org/viewvc?view=revision&revision=1425576 LUCENE-4638 , SOLR-3180 : revert for now (try using the IW's writeLock to unlock)
          Hide
          Steve Rowe added a comment -

          Bulk move 4.4 issues to 4.5 and 5.0

          Show
          Steve Rowe added a comment - Bulk move 4.4 issues to 4.5 and 5.0
          Hide
          Uwe Schindler added a comment -

          Move issue to Lucene 4.9.

          Show
          Uwe Schindler added a comment - Move issue to Lucene 4.9.

            People

            • Assignee:
              Unassigned
              Reporter:
              Mark Miller
            • Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:

                Development