[CASSANDRA-13873] Ref bug in Scrub

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 2.2.12, 3.0.16, 3.11.2, 4.0-alpha1, 4.0
• Component/s: Legacy/Tools
• Severity: Normal
• Assignee: Marcus Eriksson
• Reporter: T Jake Luciani

Authors

Marcus Eriksson

Reviewers

Joel Knighton

• Created: 14/Sep/17 19:59
• Updated: 15/May/20 08:00
• Resolved: 11/Dec/17 08:08

Description

I'm hitting a Ref bug when many scrubs run against a node. This doesn't happen on 3.0.X. I'm not sure if/if not this happens with compactions too but I suspect it does.

I'm not seeing any Ref leaks or double frees.

To Reproduce:

./tools/bin/cassandra-stress write n=10m -rate threads=100
./bin/nodetool scrub
#Ctrl-C
./bin/nodetool scrub
#Ctrl-C
./bin/nodetool scrub
#Ctrl-C
./bin/nodetool scrub

Eventually in the logs you get:
WARN [RMI TCP Connection(4)-127.0.0.1] 2017-09-14 15:51:26,722 NoSpamLogger.java:97 - Spinning trying to capture readers [BigTableReader(path='/home/jake/workspace/cassandra2/data/data/keyspace1/standard1-2eb5c780998311e79e09311efffdcd17/mc-5-big-Data.db'), BigTableReader(path='/home/jake/workspace/cassandra2/data/data/keyspace1/standard1-2eb5c780998311e79e09311efffdcd17/mc-32-big-Data.db'), BigTableReader(path='/home/jake/workspace/cassandra2/data/data/keyspace1/standard1-2eb5c780998311e79e09311efffdcd17/mc-31-big-Data.db'), BigTableReader(path='/home/jake/workspace/cassandra2/data/data/keyspace1/standard1-2eb5c780998311e79e09311efffdcd17/mc-29-big-Data.db'), BigTableReader(path='/home/jake/workspace/cassandra2/data/data/keyspace1/standard1-2eb5c780998311e79e09311efffdcd17/mc-27-big-Data.db'), BigTableReader(path='/home/jake/workspace/cassandra2/data/data/keyspace1/standard1-2eb5c780998311e79e09311efffdcd17/mc-26-big-Data.db'), BigTableReader(path='/home/jake/workspace/cassandra2/data/data/keyspace1/standard1-2eb5c780998311e79e09311efffdcd17/mc-20-big-Data.db')],
released: [BigTableReader(path='/home/jake/workspace/cassandra2/data/data/keyspace1/standard1-2eb5c780998311e79e09311efffdcd17/mc-5-big-Data.db')],

This released table has a selfRef of 0 but is in the Tracker

Issue Links

• duplicates: CASSANDRA-11155 Scrub/Cleanup block snapshots