Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-9107

More accurate row count estimates



    • Improvement
    • Status: Resolved
    • Normal
    • Resolution: Fixed
    • 2.1.6, 2.2.0 rc1
    • None
    • None


      Currently the estimated row count from cfstats is the sum of the number of rows in all the sstables. This becomes very inaccurate with wide rows or heavily updated datasets since the same partition would exist in many sstables. In example:

      create KEYSPACE test WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 1};
      create TABLE wide (key text PRIMARY KEY , value text) WITH compaction = {'class': 'SizeTieredCompactionStrategy', 'min_threshold': 30, 
      'max_threshold': 100} ;
      insert INTO wide (key, value) VALUES ('key', 'value');
      // flush
      // cfstats output: Number of keys (estimate): 1  (128 in older version from index)
      insert INTO wide (key, value) VALUES ('key', 'value');
      // flush
      // cfstats output: Number of keys (estimate): 2  (256 in older version from index)
      ... etc

      previously it used the index but it still did it per sstable and summed them up which became inaccurate as there are more sstables (just by much worse). With new versions of sstables we can merge the cardinalities to resolve this with a slight hit to accuracy in the case of every sstable having completely unique partitions.

      Furthermore I think it would be pretty minimal effort to include the number of rows in the memtables to this count. We wont have the cardinality merging between memtables and sstables but I would consider that a relatively minor negative.


        1. 9107-cassandra2-1.patch
          6 kB
          Chris Lohfink
        2. 9107-v2.txt
          6 kB
          Sam Tunnicliffe



            cnlwsu Chris Lohfink
            cnlwsu Chris Lohfink
            Chris Lohfink
            Sam Tunnicliffe
            0 Vote for this issue
            5 Start watching this issue