[CASSANDRA-10697] Leak detected while running offline scrub

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 2.1.14, 2.2.6, 3.0.4, 3.4
• Component/s: Legacy/Tools

Environment

C* 2.1.9 on Debian Wheezy

• Severity: Normal
• Assignee: Marcus Eriksson
• Reporter: mlowicki

Authors

Marcus Eriksson

Reviewers

Stefania Alborghetti

• Created: 13/Nov/15 07:33
• Updated: 16/Apr/19 09:30
• Resolved: 11/Feb/16 07:31

Description

I got couple of those:

ERROR 05:09:15 LEAK DETECTED: a reference (org.apache.cassandra.utils.concurrent.Ref$State@3b60e162) to class org.apache.cassandra.io.sstable.SSTableReader$InstanceTidier@1433208674:/var/lib/cassandra/data/sync/entity2-e24b5040199b11e5a30f75bb514ae072/sync-entity2-ka-405434 was not released before the reference was garbage collected

and then:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

        at org.apache.cassandra.io.compress.CompressedRandomAccessReader.decompressChunk(CompressedRandomAccessReader.java:99)

        at org.apache.cassandra.io.compress.CompressedRandomAccessReader.reBuffer(CompressedRandomAccessReader.java:81)

        at org.apache.cassandra.io.util.RandomAccessReader.read(RandomAccessReader.java:353)

        at java.io.RandomAccessFile.readFully(RandomAccessFile.java:444)

        at java.io.RandomAccessFile.readFully(RandomAccessFile.java:424)

        at org.apache.cassandra.io.util.RandomAccessReader.readBytes(RandomAccessReader.java:378)

        at org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:348)

        at org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:327)

        at org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:397)

        at org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:381)

        at org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:75)

        at org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:52)

        at org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:46)

        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)

        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)

        at org.apache.cassandra.io.sstable.SSTableIdentityIterator.hasNext(SSTableIdentityIterator.java:120)

        at org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:202)

        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)

        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)

        at com.google.common.collect.Iterators$7.computeNext(Iterators.java:645)

        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)

        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)

        at org.apache.cassandra.db.ColumnIndex$Builder.buildForCompaction(ColumnIndex.java:165)

        at org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:121)

        at org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:192)

        at org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:127)

        at org.apache.cassandra.io.sstable.SSTableRewriter.tryAppend(SSTableRewriter.java:158)

        at org.apache.cassandra.db.compaction.Scrubber.scrub(Scrubber.java:220)

        at org.apache.cassandra.tools.StandaloneScrubber.main(StandaloneScrubber.java:116)