[CASSANDRA-9405] AssertionError: Data component is missing

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 2.2.0 beta 1, 3.0 alpha 1
• Severity: Normal
• Assignee: Marcus Eriksson
• Reporter: Tom Hobbs

Authors

Marcus Eriksson

Reviewers

Sam Tunnicliffe

• Created: 15/May/15 20:23
• Updated: 16/Apr/19 09:31
• Resolved: 18/May/15 13:17

Description

In a counter dtest that drains and restarts nodes several times, we're occasionally seeing errors in the logs after restarting like the following:

ERROR [SSTableBatchOpen:2] 2015-05-15 16:54:41,087 DebuggableThreadPoolExecutor.java:242 - Error in ThreadPoolExecutor
java.lang.AssertionError: Data component is missing for sstable/tmp/dtest-r8mRyB/test/node2/data/system/sstable_activity-5a1ff267ace03f128563cfae6103c65e/la-5-big
    at org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:414) ~[main/:na]
    at org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:346) ~[main/:na]
    at org.apache.cassandra.io.sstable.format.SSTableReader$4.run(SSTableReader.java:476) ~[main/:na]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) ~[na:1.7.0_76]
    at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_76]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_76]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_76]
    at java.lang.Thread.run(Thread.java:745) [na:1.7.0_76]

Looking at the node's logs (attached), the sstable_activity table started a compaction just prior to the node being drained, and la-5-big is probably the partial compaction result.

I set the fix version as 2.2, but the problem may exist in earlier C* versions.