[CASSANDRA-13785] Compaction fails for SSTables with large number of keys

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Duplicate
• Component/s: Local/Compaction
• Severity: Normal
• Assignee: Jay Zhuang
• Reporter: Jay Zhuang

Authors

Jay Zhuang

• Created: 22/Aug/17 19:00
• Updated: 16/Apr/19 09:30
• Resolved: 29/Aug/17 21:13

Description

Every a few minutes there're "LEAK DTECTED" messages in the log:

ERROR [Reference-Reaper:1] 2017-08-18 17:18:40,357 Ref.java:223 - LEAK DETECTED: a reference (org.apache.cassandra.utils.concurrent.Ref$State@3ed22d7) to class org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$Tidy@1022568824:[Memory@[0..159b6ba4), Memory@[0..d8123468)] was not released before the reference was garbage collected
ERROR [Reference-Reaper:1] 2017-08-18 17:20:49,693 Ref.java:223 - LEAK DETECTED: a reference (org.apache.cassandra.utils.concurrent.Ref$State@6470405b) to class org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$Tidy@97898152:[Memory@[0..159b6ba4), Memory@[0..d8123468)] was not released before the reference was garbage collected
ERROR [Reference-Reaper:1] 2017-08-18 17:22:38,519 Ref.java:223 - LEAK DETECTED: a reference (org.apache.cassandra.utils.concurrent.Ref$State@6fc4af5f) to class org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$Tidy@1247404854:[Memory@[0..159b6ba4), Memory@[0..d8123468)] was not released before the reference was garbage collected

Debugged the issue and found it's triggered by failed compactions, if the compacted SSTable has more than 51m Integer.MAX_VALUE / 40) keys, it will fail to create the IndexSummary: IndexSummary:84.
Cassandra compaction tried to compact every a few minutes and keeps failing.

The root cause is while creating SafeMemoryWriter with > Integer.MAX_VALUE space, it returns the tailing Integer.MAX_VALUE space SafeMemoryWriter.java:83, which makes the first entries.length() not 0. So the assert fails here: IndexSummary:84

Issue Links

• duplicates: CASSANDRA-12014 IndexSummary > 2G causes an assertion error