[CASSANDRA-14103] Fix potential race during compaction strategy reload

• Type: Bug
• Status: Resolved
• Priority: Urgent
• Resolution: Fixed
• Fix Version/s: 3.11.9, 4.0-beta3, 4.0
• Component/s: Local/Compaction
• Severity: Critical
• Complexity: Normal
• Since Version: 3.11.2

Source Control Link

https://github.com/apache/cassandra/commit/f41ea9fb14936bca4aeea0ab2bf6d55c51f37f6a

Test and Documentation Plan

new unit tests, cci run

• Assignee: Marcus Eriksson
• Reporter: Paulo Motta

Authors

Marcus Eriksson

Reviewers

Blake Eggleston

• Created: 08/Dec/17 17:59
• Updated: 02/Apr/23 12:06
• Resolved: 16/Sep/20 09:14

Description

When the compaction strategies are reloaded after disk boundary changes (CASSANDRA-13948), it's possible that a recently finished SSTable is added twice to the compaction strategy: once when the compaction strategies are reloaded due to the disk boundary change (maybeReloadDiskBoundarie), and another when the CompactionStrategyManager is processing the SSTableAddedNotification.

This should be quite unlikely because a compaction must finish as soon as the disk boundary changes, and even if it happens most compaction strategies would not be affected by it since they deduplicate sstables internally, but we should protect against such scenario.

For more context see this comment from Marcus.

Issue Links

• fixes: CASSANDRA-14642 Document lock handling in CompactionStrategyManager
• is duplicated by: CASSANDRA-14826 cassandra spinning forever on 1 thread while initializing keyspace
• is duplicated by: CASSANDRA-15242 Race condition between flushing and compaction stalls compaction indefinitely
• relates to: CASSANDRA-13948 Reload compaction strategies when JBOD disk boundary changes