Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-14647

Reading cardinality from Statistics.db failed

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Low
    • Resolution: Fixed
    • 4.0-alpha1, 4.0
    • Observability/Metrics
    • None
    • Low
    • Hide

      adds test case

      Show
      adds test case

    Description

      There is some issue with sstable metadata which is visible in system.log, the messages says:

      WARN  [Thread-6] 2018-07-25 07:12:47,928 SSTableReader.java:249 - Reading cardinality from Statistics.db failed for /opt/data/disk5/data/keyspace/table/mc-big-Data.db.

      Although there is no such file.

      The message has appeared after i've changed the compaction strategy from SizeTiered to Leveled. Compaction strategy has been changed region by region (total 3 regions) and it has coincided with the double client write traffic increase.
      I have tried to run nodetool scrub to rebuilt the sstable, but that does not fix the issue.

      So very hard to define the steps to reproduce, probably it will be:

      1. run stress tool with write traffic
      2. under load change compaction strategy from SireTiered to Leveled for the bunch of hosts
      3. add more write traffic

      Reading the code it is said that if this metadata is broken, then "estimating the keys will be done using index summary". 
      https://github.com/apache/cassandra/blob/cassandra-3.0.17/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java#L247
       

      Attachments

        1. 14647-trunk-1.patch
          11 kB
          Venkata Harikrishna Nukala
        2. cassandra_compaction_pending_tasks_7days.png
          1003 kB
          Vitali Djatsuk

        Activity

          People

            n.v.harikrishna Venkata Harikrishna Nukala
            nezdali Vitali Djatsuk
            Venkata Harikrishna Nukala
            Marcus Eriksson
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: