[CASSANDRA-8904] Compaction pending tasks is 9 but no compaction is running

• Type: Bug
• Status: Resolved
• Priority: Low
• Resolution: Fixed
• Fix Version/s: 2.0.15
• Labels: compaction

Environment

Linux 2.6.32-431.5.1.el6.x86_64 #1 SMP Wed Feb 12 00:41:43 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

java version "1.7.0_71"
Java(TM) SE Runtime Environment (build 1.7.0_71-b14)
Java HotSpot(TM) 64-Bit Server VM (build 24.71-b01, mixed mode)

[cqlsh 3.1.8 | Cassandra 1.2.18 | CQL spec 3.0.5 | Thrift protocol 19.36.2]

• Severity: Low
• Assignee: Carl Yeksigian
• Reporter: Zdenek Ott

Authors

Carl Yeksigian

Reviewers

Marcus Eriksson

• Created: 04/Mar/15 09:57
• Updated: 16/Apr/19 09:31
• Resolved: 17/Mar/15 10:20

Description

I have found mismatch of compaction requirement computation in LeveldManifest.java

—
In getCompactionCandidates() method is used following expression:
double score = (double) SSTableReader.getTotalBytes(remaining) / (double)maxBytesForLevel;
And compaction is triggered when score>1.001

In our situation 671758260447/(10^4*64*1024*1024) = 1.00099

—
in getEstimatedTasks() method is used different expression:

estimated[i] = Math.max(0L, SSTableReader.getTotalBytes(sstables) - maxBytesForLevel) / maxSSTableSizeInBytes

In our situation (671758260447-(10^4*64*1024*1024))/(64*1024*1024) = 9.97812 implicitly converted to long is 9

—
Here are debug log parts and column family create command:

DEBUG [CompactionExecutor:1695] 2015-03-04 10:31:46,173 LeveledManifest.java (line 384) L0 contains 2 SSTables (24467649 bytes) in Manifest@160946690
DEBUG [CompactionExecutor:1695] 2015-03-04 10:31:46,173 LeveledManifest.java (line 384) L1 contains 10 SSTables (659509359 bytes) in Manifest@160946690
DEBUG [CompactionExecutor:1695] 2015-03-04 10:31:46,173 LeveledManifest.java (line 384) L2 contains 105 SSTables (6670197371 bytes) in Manifest@160946690
DEBUG [CompactionExecutor:1695] 2015-03-04 10:31:46,173 LeveledManifest.java (line 384) L3 contains 1043 SSTables (67139124061 bytes) in Manifest@160946690
DEBUG [CompactionExecutor:1695] 2015-03-04 10:31:46,174 LeveledManifest.java (line 384) L4 contains 10410 SSTables (671758260447 bytes) in Manifest@160946690
DEBUG [CompactionExecutor:1695] 2015-03-04 10:31:46,174 LeveledManifest.java (line 384) L5 contains 75 SSTables (5019585316 bytes) in Manifest@160946690

DEBUG [CompactionExecutor:1695] 2015-03-04 10:31:46,191 LeveledManifest.java (line 346) Compaction score for level 5 is 7.479764991998672E-4
DEBUG [CompactionExecutor:1695] 2015-03-04 10:31:46,196 LeveledManifest.java (line 346) Compaction score for level 4 is 1.0009978122219443
DEBUG [CompactionExecutor:1695] 2015-03-04 10:31:46,197 LeveledManifest.java (line 346) Compaction score for level 3 is 1.0004509100466967
DEBUG [CompactionExecutor:1695] 2015-03-04 10:31:46,197 LeveledManifest.java (line 346) Compaction score for level 2 is 0.9939368622004986
DEBUG [CompactionExecutor:1695] 2015-03-04 10:31:46,197 LeveledManifest.java (line 346) Compaction score for level 1 is 0.9827455267310142
DEBUG [CompactionExecutor:1695] 2015-03-04 10:31:46,197 LeveledManifest.java (line 346) Compaction score for level 0 is 0.0883796289563179

DEBUG [RMI TCP Connection(3047)-77.234.40.29] 2015-03-04 10:32:18,093 LeveledManifest.java (line 670) Estimating [0, 0, 0, 0, 9, 0, 0, 0, 0] compactions to do for fr.sha

CREATE TABLE sha (
"KEY" blob,
column1 blob,
value blob,
PRIMARY KEY ("KEY", column1)
) WITH COMPACT STORAGE AND
bloom_filter_fp_chance=0.100000 AND
caching='NONE' AND
comment='' AND
dclocal_read_repair_chance=0.000000 AND
gc_grace_seconds=172800 AND
read_repair_chance=0.000000 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
compaction=

{'sstable_size_in_mb': '64', 'class': 'LeveledCompactionStrategy'}

AND
compression={};