Uploaded image for project: 'Tephra'
  1. Tephra
  2. TEPHRA-235

Flushes and compactions can remove committed data

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.11.0-incubating, 0.12.0-incubating
    • Fix Version/s: 0.13.0-incubating
    • Component/s: core
    • Labels:
      None

      Description

      Transaction co-processor uses the transaction snapshot generated by the Transaction Manager to remove deleted and invalid cells during HBase compactions and flushes. The co-processor expects the invalid list in the transaction snapshot to be sorted. However, the invalid list is not sorted during a snapshot creation in the Transaction Manager. This leads to invalid transactions becoming visible during compactions, and thus committed transactions with write pointers smaller than the invalid transaction will get removed.

      We missed this piece when fixing TEPHRA-223.

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/incubator-tephra/pull/44

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/incubator-tephra/pull/44
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user poornachandra commented on a diff in the pull request:

          https://github.com/apache/incubator-tephra/pull/44#discussion_r123363497

          — Diff: tephra-core/src/test/java/org/apache/tephra/snapshot/SnapshotCodecTest.java —
          @@ -375,4 +386,17 @@ private void assertTransactionVisibilityStateEquals(TransactionVisibilityState e
          Assert.assertEquals(expected.getInProgress(), input.getInProgress());
          Assert.assertEquals(expected.getInvalid(), input.getInvalid());
          }
          +
          + private TransactionSnapshot assertSorted(TransactionSnapshot txSnapshot) {
          + Collection<Long> invalidTxList = txSnapshot.getInvalid();
          + if (invalidTxList.isEmpty())

          { + return txSnapshot; + }

          + Long previousLong = Iterables.get(invalidTxList, 0);
          + for (Long aLong : invalidTxList) {
          + Assert.assertTrue(aLong >= previousLong);
          — End diff –

          Would be good to say sort error here

          Show
          githubbot ASF GitHub Bot added a comment - Github user poornachandra commented on a diff in the pull request: https://github.com/apache/incubator-tephra/pull/44#discussion_r123363497 — Diff: tephra-core/src/test/java/org/apache/tephra/snapshot/SnapshotCodecTest.java — @@ -375,4 +386,17 @@ private void assertTransactionVisibilityStateEquals(TransactionVisibilityState e Assert.assertEquals(expected.getInProgress(), input.getInProgress()); Assert.assertEquals(expected.getInvalid(), input.getInvalid()); } + + private TransactionSnapshot assertSorted(TransactionSnapshot txSnapshot) { + Collection<Long> invalidTxList = txSnapshot.getInvalid(); + if (invalidTxList.isEmpty()) { + return txSnapshot; + } + Long previousLong = Iterables.get(invalidTxList, 0); + for (Long aLong : invalidTxList) { + Assert.assertTrue(aLong >= previousLong); — End diff – Would be good to say sort error here
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user anwar6953 opened a pull request:

          https://github.com/apache/incubator-tephra/pull/44

          TEPHRA-235 Ensure that TransactionSnapshot always have a sorted invalid transaction list

          TEPHRA-235(https://issues.apache.org/jira/browse/TEPHRA-235) Ensure that TransactionSnapshot always have a sorted invalid transaction list.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/anwar6953/incubator-tephra bugfix_release/TEPHRA-235

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/incubator-tephra/pull/44.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #44


          commit 9d482be0aaf58b296931c440b393e59976c38eab
          Author: Ali Anwar <anwar1@berkeley.edu>
          Date: 2017-06-21T01:15:56Z

          TEPHRA-235 Ensure that TransactionSnapshot always have a sorted invalid transaction list.


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user anwar6953 opened a pull request: https://github.com/apache/incubator-tephra/pull/44 TEPHRA-235 Ensure that TransactionSnapshot always have a sorted invalid transaction list TEPHRA-235 ( https://issues.apache.org/jira/browse/TEPHRA-235 ) Ensure that TransactionSnapshot always have a sorted invalid transaction list. You can merge this pull request into a Git repository by running: $ git pull https://github.com/anwar6953/incubator-tephra bugfix_release/ TEPHRA-235 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-tephra/pull/44.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #44 commit 9d482be0aaf58b296931c440b393e59976c38eab Author: Ali Anwar <anwar1@berkeley.edu> Date: 2017-06-21T01:15:56Z TEPHRA-235 Ensure that TransactionSnapshot always have a sorted invalid transaction list.

            People

            • Assignee:
              anwar1 Ali Anwar
              Reporter:
              poornachandra Poorna Chandra
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development