Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-10965

Shadowable tombstones can continue to shadow view results when timestamps match

    Details

      Description

      I've attached a script which reproduces the issue. The first time we insert with TIMESTAMP 2, we are inserting a new row which has the same timestamp as the previous shadow tombstone, and it continues to be shadowed by that tombstone because we shadow values with the same timestamp.

      1. shadow-ts.cql
        0.9 kB
        Carl Yeksigian

        Issue Links

          Activity

          Hide
          firstprayer Taiyuan Zhang added a comment - - edited

          I ran the script, and here is the output:

          cqlsh:mykeyspace> SELECT c, k, val FROM base; SELECT c, k, val FROM mv_reuse;
          
           c | k | val
          ---+---+-----
           0 | 1 |   1
          
          (1 rows)
          
           c | k | val
          ---+---+-----
           0 | 1 |   1
          
          (1 rows)
          cqlsh:mykeyspace> UPDATE base USING TIMESTAMP 1 SET c = 1 WHERE k = 1;
          cqlsh:mykeyspace> SELECT c, k, val FROM base; SELECT c, k, val FROM mv_reuse;
          
           c | k | val
          ---+---+-----
           1 | 1 |   1
          
          (1 rows)
          
           c | k | val
          ---+---+-----
          
          (0 rows)
          

          So the problem is: after the update using the same timestamp, the row is not shown when query from the materialized view?

          Show
          firstprayer Taiyuan Zhang added a comment - - edited I ran the script, and here is the output: cqlsh:mykeyspace> SELECT c, k, val FROM base; SELECT c, k, val FROM mv_reuse; c | k | val ---+---+----- 0 | 1 | 1 (1 rows) c | k | val ---+---+----- 0 | 1 | 1 (1 rows) cqlsh:mykeyspace> UPDATE base USING TIMESTAMP 1 SET c = 1 WHERE k = 1; cqlsh:mykeyspace> SELECT c, k, val FROM base; SELECT c, k, val FROM mv_reuse; c | k | val ---+---+----- 1 | 1 | 1 (1 rows) c | k | val ---+---+----- (0 rows) So the problem is: after the update using the same timestamp, the row is not shown when query from the materialized view?
          Hide
          carlyeks Carl Yeksigian added a comment -

          Taiyuan Zhang: Yes, that's the issue here; while we get a value from the base, we have hidden it in the view because of the tombstones that we generated for the previous value.

          Show
          carlyeks Carl Yeksigian added a comment - Taiyuan Zhang : Yes, that's the issue here; while we get a value from the base, we have hidden it in the view because of the tombstones that we generated for the previous value.
          Hide
          firstprayer Taiyuan Zhang added a comment -

          Got it. Are you currently working on this? If not, I'm interested in digging into this.

          Show
          firstprayer Taiyuan Zhang added a comment - Got it. Are you currently working on this? If not, I'm interested in digging into this.
          Hide
          carlyeks Carl Yeksigian added a comment -

          Taiyuan Zhang: I haven't started working on this, but it is a pretty high priority bug, so we'd like to get this in with the 3.3 bug fix release, which will happen in early February, so this will have to be fixed sometime in the next 2-3 weeks. Does that timeline make sense for you?

          Show
          carlyeks Carl Yeksigian added a comment - Taiyuan Zhang : I haven't started working on this, but it is a pretty high priority bug, so we'd like to get this in with the 3.3 bug fix release, which will happen in early February, so this will have to be fixed sometime in the next 2-3 weeks. Does that timeline make sense for you?
          Hide
          firstprayer Taiyuan Zhang added a comment -

          I haven't started looking into this so I don't know how complicated the problem is, but the time seems to fit into my calendar. I'll look into this in recent 1 or 2 days to better understand the problem and give a better estimation.

          Show
          firstprayer Taiyuan Zhang added a comment - I haven't started looking into this so I don't know how complicated the problem is, but the time seems to fit into my calendar. I'll look into this in recent 1 or 2 days to better understand the problem and give a better estimation.
          Hide
          carlyeks Carl Yeksigian added a comment -

          When we resolve 2 cells with conflicting timestamps for the base table, we use the value to determine which cell wins. However, the view has no information about this conflict or the resolution.

          Adding another field to the deletion would solve this; it would indicate whether the tombstone should shadow a value with the same time since a resolution in the same time resolved the values, or whether it should only apply to values which are before the tombstone, thus a tie with the tombstone would result in the value being used, not the tombstone.

          Pushed a branch which includes this change as a "replacement" flag.

          Show
          carlyeks Carl Yeksigian added a comment - When we resolve 2 cells with conflicting timestamps for the base table, we use the value to determine which cell wins. However, the view has no information about this conflict or the resolution. Adding another field to the deletion would solve this; it would indicate whether the tombstone should shadow a value with the same time since a resolution in the same time resolved the values, or whether it should only apply to values which are before the tombstone, thus a tie with the tombstone would result in the value being used, not the tombstone. Pushed a branch which includes this change as a "replacement" flag.
          Hide
          slebresne Sylvain Lebresne added a comment -

          I don't think that change is sufficient in practice, but the why requires a bit of context and it's all in CASSANDRA-11500. Basically, I like to think of this case is a nasty special case of CASSANDRA-11500 so it might be as simple to just handle both side of the project in CASSANDRA-11500.

          I'll note that the repro script of this ticket actually doesn't fail on CASSANDRA-11475 due to difference with how the deletion timestamp is picked, but it's easy to add one more step to this test to reproduce a similar problem. So CASSANDRA-11475 doesn't really solve this, it just make it a little harder to run into.

          Show
          slebresne Sylvain Lebresne added a comment - I don't think that change is sufficient in practice, but the why requires a bit of context and it's all in CASSANDRA-11500 . Basically, I like to think of this case is a nasty special case of CASSANDRA-11500 so it might be as simple to just handle both side of the project in CASSANDRA-11500 . I'll note that the repro script of this ticket actually doesn't fail on CASSANDRA-11475 due to difference with how the deletion timestamp is picked, but it's easy to add one more step to this test to reproduce a similar problem. So CASSANDRA-11475 doesn't really solve this, it just make it a little harder to run into.
          Hide
          KurtG Kurt Greaves added a comment -

          Resolving as duplicate, will solve in CASSANDRA-11500

          Show
          KurtG Kurt Greaves added a comment - Resolving as duplicate, will solve in CASSANDRA-11500

            People

            • Assignee:
              Unassigned
              Reporter:
              carlyeks Carl Yeksigian
              Reviewer:
              Sylvain Lebresne
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development