Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-15918

materialized view rebuild automatically after drop multiple views



    • Bug
    • Status: Open
    • Normal
    • Resolution: Unresolved
    • None
    • None
    • Degradation
    • Low
    • Normal
    • User Report
    • All
    • None



      Cassandra version: 3.0.12

      Our cassandra cluster has 9 host for DC1 and 3 host for DC2,

      each host :

      node memory disk
      DC1 1 256 GB 5*788GB
      DC1 2 256 GB 5*788GB
      DC1 3 256 GB 5*788GB
      DC1 4 256 GB 5*788GB
      DC1 5 256 GB 5*788GB
      DC1 6 256 GB 5*788GB
      DC1 7 512 GB 5 TB
      DC1 8 512 GB 5 TB
      DC1 9 512 GB 5 TB
      DC2 1 256 GB 8*788GB
      DC2 2 256 GB 8*788GB
      DC2 3 256 GB 8*788GB

      by using nodetool status, node load in DC1 is about 1.5 TB, node load in DC2 is about 4 TB

      QPS: 270


      Problem we met:

      In DC1 1 node, enter the cql command line and execute command like following in sametime:

      "drop materialized view if exists view1; 

      drop materialized view if exists view2;

      drop materialized view if exists view3;

      drop materialized view if exists view4;"

      after a while, command line display warning like "schema version mismatch detected..." (sorry we cannot find the exact output for that time)

      After that we find view files in node: "DC1 7" hasn't be deleted yet.

      at this moment, we find performance of cluster drop sharp, the cluster almost stop response to any request.

      by runing:  select * from system.views_builds_in_progress;

      we can see several views were building.

      then we execte:

      1,   nodetool stop VIEW_BUILD  in each node

      2, in cql: delete from system.views_builds_in_progress where view_name=XXXX

      3, rolling restart cassandra nodes


      about an hours later, performance increase to normal.


      Why this happen?

      How to avoid this problem?

      Any better way to deal with this problem?




            Unassigned Unassigned
            mark404 chonghao li
            0 Vote for this issue
            4 Start watching this issue