[CASSANDRA-8905] IllegalArgumentException in compaction of -ic- file after upgrade to 2.0.12

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Not A Problem
• Fix Version/s: 2.0.15
• Severity: Normal
• Assignee: Unassigned
• Reporter: Erik Forsberg
• Created: 04/Mar/15 13:08
• Updated: 16/Apr/19 09:31
• Resolved: 03/Apr/15 16:40

Description

After upgrade from 1.2.18 to 2.0.12, I've started to get exceptions like:

ERROR [CompactionExecutor:1149] 2015-03-04 11:48:46,045 CassandraDaemon.java (line 199) Exception in thread Thread[CompactionExecutor:1149,1,main]
java.lang.IllegalArgumentException: Illegal Capacity: -2147483648
        at java.util.ArrayList.<init>(ArrayList.java:142)
        at org.apache.cassandra.db.SuperColumns$SCIterator.next(SuperColumns.java:182)
        at org.apache.cassandra.db.SuperColumns$SCIterator.next(SuperColumns.java:194)
        at org.apache.cassandra.db.SuperColumns$SCIterator.next(SuperColumns.java:138)
        at org.apache.cassandra.io.sstable.SSTableIdentityIterator.getColumnFamilyWithColumns(SSTableIdentityIterator.java:186)
        at org.apache.cassandra.db.compaction.PrecompactedRow.merge(PrecompactedRow.java:98)
        at org.apache.cassandra.db.compaction.PrecompactedRow.<init>(PrecompactedRow.java:85)
        at org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:196)
        at org.apache.cassandra.db.compaction.CompactionIterable$Reducer.getReduced(CompactionIterable.java:74)
        at org.apache.cassandra.db.compaction.CompactionIterable$Reducer.getReduced(CompactionIterable.java:55)
        at org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:115)
        at org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:98)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:161)
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
        at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
        at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
        at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:198)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)

I've identified which sstable is causing this, it's an ic format sstable, i.e. something written before the upgrade. I can repeat with forceUserDefinedCompaction.

Running upgradesstables also causes the same exception.

Scrub helps, but skips a row as incorrect.

I can share the sstable privately if it helps.