Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-10346

Create tool to repair inconsistencies between base tables and materialized views when dataloss ocurrs

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Fix Version/s: 4.x
    • Component/s: None
    • Labels:
      None

      Description

      As mentioned in CASSANDRA-10230, you can have inconsistencies between base and view if you have data loss on all ack'd replicas in the base table.

      We should add a way to identify and ideally fix/drop discrepancies between the base and view replicas.

      I think implementing this efficiently will not be so easy but it should work like repair in theory.

        Activity

        Hide
        carlyeks Carl Yeksigian added a comment -

        Here are the difficulties that I've thought of so far (there are probably others, these just give me the most pause):

        • To do a merkle tree repair, all the base table values will need to be transformed first before calculation since the values are different between base and view
        • Inserting new values won't clean up old values, so just restreaming the values wouldn't help with this operation as it wouldn't remove any values that were lost from the data loss on the base table
        • A base replica would have data on all of the view replicas (in the same DC), and would need to exchange merkle trees with all of them, which would not be like repair which only needs to exchange with RF-1 replicas

        The current workaround is to drop and recreate the MV.

        An idea I had early was to run a build operation into a table with a different internal table ID and swap the view after it is built, but it would require knowing when the whole ring is done building the view.

        Show
        carlyeks Carl Yeksigian added a comment - Here are the difficulties that I've thought of so far (there are probably others, these just give me the most pause): To do a merkle tree repair, all the base table values will need to be transformed first before calculation since the values are different between base and view Inserting new values won't clean up old values, so just restreaming the values wouldn't help with this operation as it wouldn't remove any values that were lost from the data loss on the base table A base replica would have data on all of the view replicas (in the same DC), and would need to exchange merkle trees with all of them, which would not be like repair which only needs to exchange with RF-1 replicas The current workaround is to drop and recreate the MV. An idea I had early was to run a build operation into a table with a different internal table ID and swap the view after it is built, but it would require knowing when the whole ring is done building the view.
        Hide
        michaelsembwever mck added a comment -

        Bumping to fix version 4.x, as 3.11.0 is a bug-fix only release.
          ref https://s.apache.org/EHBy

        Show
        michaelsembwever mck added a comment - Bumping to fix version 4.x, as 3.11.0 is a bug-fix only release.   ref https://s.apache.org/EHBy

          People

          • Assignee:
            Unassigned
            Reporter:
            tjake T Jake Luciani
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:

              Development