[CASSANDRA-12394] Node crashes during compaction with assertion error checking length of entries for IndexSummaryBuilder

• Type: Bug
• Status: Resolved
• Priority: Urgent
• Resolution: Duplicate
• Component/s: Local/Compaction
• Severity: Critical
• Since Version: 2.1.5
• Assignee: Unassigned
• Reporter: Ryan Svihla
• Created: 05/Aug/16 15:22
• Updated: 16/Apr/19 09:30
• Resolved: 05/Aug/16 20:31

Description

This limit appears to be arbitrarily Integer.MAX the rest of the values are all referencing a type of long. Worse when this is encountered the node crashes. Seems to be introduced in
https://issues.apache.org/jira/browse/CASSANDRA-8757

Sstables are 350-440gb

Relevant code is https://github.com/apache/cassandra/blob/cassandra-2.1/src/java/org/apache/cassandra/io/sstable/IndexSummaryBuilder.java#L171

stack trace:

[CompactionExecutor:47340] 2016-07-17 04:55:08,966 CassandraDaemon.java:229 - Exception in thread Thread[CompactionExecutor:47340,1,main]
java.lang.AssertionError: null
at org.apache.cassandra.io.sstable.IndexSummaryBuilder.maybeAddEntry(IndexSummaryBuilder.java:171) ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at org.apache.cassandra.io.sstable.SSTableWriter$IndexWriter.append(SSTableWriter.java:634) ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at org.apache.cassandra.io.sstable.SSTableWriter.afterAppend(SSTableWriter.java:179) ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:205) ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:126) ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:197) ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:73) ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59) ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:263) ~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) ~[na:1.7.0_51]
at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_51]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_51]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_51]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]

Issue Links

• duplicates: CASSANDRA-12014 IndexSummary > 2G causes an assertion error