Derby
  1. Derby
  2. DERBY-5258

btree post commit releases latch before committing/aborting purges, possibly allowing other operation on page

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 10.3.3.0, 10.4.2.0, 10.5.3.0, 10.6.1.0, 10.7.1.1, 10.8.1.2
    • Component/s: Store
    • Labels:
      None
    • Issue & fix info:
      High Value Fix
    • Bug behavior facts:
      Crash, Data corruption

      Description

      From code inspection found the following problem. BTreePostCommit.purgeCommittedDeletes gives up the latch in it's finally block, before the internal transaction is
      committed. The transaction is committed no sync upon return from this routine leaving a very small window when some other thread could get latch on the page and
      perform operations on the page.

      This can be a problem if for some reason the internal transaction is never committed. Purges actually return space to the page, unlike deletes. In order to backout the
      purges one must add the rows back, taking up space on the page. If another tranaction comes in before the internal transaction is committed and does inserts there may
      be no space for the backout of the purges. This is why normal delete processing only sets flags on the rows and purge processing is handled differently.

      I found this problem while debugging a database submitted as part of DERBY-5248. I believe this issue can cause the problem there, but since we have no repro have
      decided to create a new issue to target this specific problem/solution. Later can close the other issue if it can never be reproduce after the fix. In DEBY-5248 there
      are purges without a commit followed immediated by an insert in another transaction that is commited and the purge transaction is never committed. On recovery the
      system tries to abort the internal transaction and eventually trashes the page when it does not actually have enough space to abort the purge. See that issue for more
      detail.

      1. DERBY-5258_diff.txt
        11 kB
        Mike Matrigali

        Issue Links

          Activity

          Hide
          Mike Matrigali added a comment -

          Attaching proposed fix. This has not passed tests yet.
          I have not been able to reproduce the problem in my environment so the fix is just from code inspection. One change from previous comments is that this fix is to
          purgeRowLevelCommittedDeletes(). The purgeCommittedDeletes() routine is ok, as it holds a table
          level exclusive lock until end of transaction.

          I will run full set of tests before committing to trunk.

          I will try for another day to reproduce, but if I can't will probably go ahead and check in to trunk unless anyone
          feels I should not.

          Show
          Mike Matrigali added a comment - Attaching proposed fix. This has not passed tests yet. I have not been able to reproduce the problem in my environment so the fix is just from code inspection. One change from previous comments is that this fix is to purgeRowLevelCommittedDeletes(). The purgeCommittedDeletes() routine is ok, as it holds a table level exclusive lock until end of transaction. I will run full set of tests before committing to trunk. I will try for another day to reproduce, but if I can't will probably go ahead and check in to trunk unless anyone feels I should not.
          Hide
          Bryan Pendleton added a comment -

          Wow! Great find, Mike! Your description is quite clear and your theory makes sense to me.
          Your detective work is wonderful to read, thanks very much for taking the time to describe
          the sequence of actions that lead to this case.

          The patch has a few whitespace issues, I think, but other than that it looked fine to me.

          I think that the comment could read "The commit will clear the latch" rather than
          "The commit should clear the latch", as the "should" doesn't quite sound right.

          Show
          Bryan Pendleton added a comment - Wow! Great find, Mike! Your description is quite clear and your theory makes sense to me. Your detective work is wonderful to read, thanks very much for taking the time to describe the sequence of actions that lead to this case. The patch has a few whitespace issues, I think, but other than that it looked fine to me. I think that the comment could read "The commit will clear the latch" rather than "The commit should clear the latch", as the "should" doesn't quite sound right.
          Hide
          Mike Matrigali added a comment -

          All the tests passed, and I integrated Bryan's suggested comment changes. Committed to trunk:
          Sending java\engine\org\apache\derby\impl\store\access\btree\BTreePostCommit.java
          Sending java\engine\org\apache\derby\impl\store\raw\data\BasePage.java
          Sending java\engine\org\apache\derby\impl\store\raw\data\StoredPage.java
          Transmitting file data ...
          Committed revision 1132711.

          Show
          Mike Matrigali added a comment - All the tests passed, and I integrated Bryan's suggested comment changes. Committed to trunk: Sending java\engine\org\apache\derby\impl\store\access\btree\BTreePostCommit.java Sending java\engine\org\apache\derby\impl\store\raw\data\BasePage.java Sending java\engine\org\apache\derby\impl\store\raw\data\StoredPage.java Transmitting file data ... Committed revision 1132711.
          Hide
          Tim Wu added a comment -

          Hi Mike, Any chance this fix can be merged back to 10.8?

          Show
          Tim Wu added a comment - Hi Mike, Any chance this fix can be merged back to 10.8?
          Hide
          Mike Matrigali added a comment -

          i plan to backport once I verify that tests passed across all nightly's on trunk and I run tests on 10.8.

          Show
          Mike Matrigali added a comment - i plan to backport once I verify that tests passed across all nightly's on trunk and I run tests on 10.8.
          Hide
          Mike Matrigali added a comment -

          backported change from trunk to 10.8 branch:

          s108_ibm16:11>svn commit

          Sending java\engine\org\apache\derby\impl\store\access\btree\BTreePostCommit.java
          Sending java\engine\org\apache\derby\impl\store\raw\data\BasePage.java
          Sending java\engine\org\apache\derby\impl\store\raw\data\StoredPage.java
          Transmitting file data ...
          Committed revision 1133470.

          Show
          Mike Matrigali added a comment - backported change from trunk to 10.8 branch: s108_ibm16:11>svn commit Sending java\engine\org\apache\derby\impl\store\access\btree\BTreePostCommit.java Sending java\engine\org\apache\derby\impl\store\raw\data\BasePage.java Sending java\engine\org\apache\derby\impl\store\raw\data\StoredPage.java Transmitting file data ... Committed revision 1133470.
          Hide
          Mike Matrigali added a comment -

          backported the fix from trunk to 10.7 branch.

          s107_ibm16:5>svn commit

          Sending java\engine\org\apache\derby\impl\store\access\btree\BTreePostCommit.java
          Sending java\engine\org\apache\derby\impl\store\raw\data\BasePage.java
          Sending java\engine\org\apache\derby\impl\store\raw\data\StoredPage.java
          Transmitting file data ...
          Committed revision 1134092.

          Show
          Mike Matrigali added a comment - backported the fix from trunk to 10.7 branch. s107_ibm16:5>svn commit Sending java\engine\org\apache\derby\impl\store\access\btree\BTreePostCommit.java Sending java\engine\org\apache\derby\impl\store\raw\data\BasePage.java Sending java\engine\org\apache\derby\impl\store\raw\data\StoredPage.java Transmitting file data ... Committed revision 1134092.
          Hide
          Mike Matrigali added a comment -

          back ported to 10.6 branch, with minor conflict fix:

          s106_ibm16:31>svn commit

          Sending java\engine\org\apache\derby\impl\store\access\btree\BTreePostCommit.java
          Sending java\engine\org\apache\derby\impl\store\raw\data\BasePage.java
          Sending java\engine\org\apache\derby\impl\store\raw\data\StoredPage.java
          Transmitting file data ...
          Committed revision 1134098.

          Show
          Mike Matrigali added a comment - back ported to 10.6 branch, with minor conflict fix: s106_ibm16:31>svn commit Sending java\engine\org\apache\derby\impl\store\access\btree\BTreePostCommit.java Sending java\engine\org\apache\derby\impl\store\raw\data\BasePage.java Sending java\engine\org\apache\derby\impl\store\raw\data\StoredPage.java Transmitting file data ... Committed revision 1134098.
          Hide
          Mike Matrigali added a comment -

          backported from trunk to 10.5 branch with minor conflict fix:

          s105_ibm16:18>svn commit

          Sending java\engine\org\apache\derby\impl\store\access\btree\BTreePostCommit.java
          Sending java\engine\org\apache\derby\impl\store\raw\data\BasePage.java
          Sending java\engine\org\apache\derby\impl\store\raw\data\StoredPage.java
          Transmitting file data ...
          Committed revision 1134103.

          Show
          Mike Matrigali added a comment - backported from trunk to 10.5 branch with minor conflict fix: s105_ibm16:18>svn commit Sending java\engine\org\apache\derby\impl\store\access\btree\BTreePostCommit.java Sending java\engine\org\apache\derby\impl\store\raw\data\BasePage.java Sending java\engine\org\apache\derby\impl\store\raw\data\StoredPage.java Transmitting file data ... Committed revision 1134103.
          Hide
          Mike Matrigali added a comment -

          backported fix from trunk to 10.4 branch, required some manual merging of conflicts.

          s104_jdk16:18>svn commit

          Sending java\engine\org\apache\derby\impl\store\access\btree\BTreePostCommit.java
          Sending java\engine\org\apache\derby\impl\store\raw\data\BasePage.java
          Sending java\engine\org\apache\derby\impl\store\raw\data\StoredPage.java
          Transmitting file data ...
          Committed revision 1135720.

          Show
          Mike Matrigali added a comment - backported fix from trunk to 10.4 branch, required some manual merging of conflicts. s104_jdk16:18>svn commit Sending java\engine\org\apache\derby\impl\store\access\btree\BTreePostCommit.java Sending java\engine\org\apache\derby\impl\store\raw\data\BasePage.java Sending java\engine\org\apache\derby\impl\store\raw\data\StoredPage.java Transmitting file data ... Committed revision 1135720.
          Hide
          Mike Matrigali added a comment -

          fixing Affects version field. This bug was introduced in 10.3 when row level purging of btree rows during post commit was added.
          Earlier releases were not affected.

          Show
          Mike Matrigali added a comment - fixing Affects version field. This bug was introduced in 10.3 when row level purging of btree rows during post commit was added. Earlier releases were not affected.
          Hide
          Mike Matrigali added a comment -

          Found issue by code inspection, fixed and backported to all affectected branches. 10.5 branch is oldest branch affected by this issue.

          Show
          Mike Matrigali added a comment - Found issue by code inspection, fixed and backported to all affectected branches. 10.5 branch is oldest branch affected by this issue.
          Hide
          Knut Anders Hatlen added a comment -

          [bulk update] Close all resolved issues that haven't been updated for more than one year.

          Show
          Knut Anders Hatlen added a comment - [bulk update] Close all resolved issues that haven't been updated for more than one year.

            People

            • Assignee:
              Mike Matrigali
              Reporter:
              Mike Matrigali
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development