[CASSANDRA-7066] Simplify (and unify) cleanup of compaction leftovers

• Type: Improvement
• Status: Resolved
• Priority: Low
• Resolution: Fixed
• Fix Version/s: 3.0 alpha 1
• Component/s: Legacy/Local Write-Read Paths
• Labels: benedict-to-commit compaction
• Assignee: Stefania Alborghetti
• Reporter: Benedict Elliott Smith

Authors

Stefania Alborghetti

Reviewers

Benedict Elliott Smith

• Created: 22/Apr/14 13:10
• Updated: 16/Apr/19 09:31
• Resolved: 19/Aug/15 12:40

Description

Currently we manage a list of in-progress compactions in a system table, which we use to cleanup incomplete compactions when we're done. The problem with this is that 1) it's a bit clunky (and leaves us in positions where we can unnecessarily cleanup completed files, or conversely not cleanup files that have been superceded); and 2) it's only used for a regular compaction - no other compaction types are guarded in the same way, so can result in duplication if we fail before deleting the replacements.

I'd like to see each sstable store in its metadata its direct ancestors, and on startup we simply delete any sstables that occur in the union of all ancestor sets. This way as soon as we finish writing we're capable of cleaning up any leftovers, so we never get duplication. It's also much easier to reason about.

Issue Links

• breaks: CASSANDRA-10105 Windows dtest 3.0: TestOfflineTools failures
• breaks: CASSANDRA-10109 Windows dtest 3.0: ttl_test.py failures
• contains: CASSANDRA-9854 Make CompactionController.maybeRefreshOverlaps public
• is related to: CASSANDRA-10006 2.1 format sstable filenames with "tmp" are not handled by 3.0
• is related to: CASSANDRA-6756 Provide option to avoid loading orphan SSTables on startup
• relates to: CASSANDRA-10421 Potential issue with LogTransaction as it only checks in a single directory for files
• relates to: CASSANDRA-9516 ViewTest.testSSTablesInBounds fails in trunk
• relates to: CASSANDRA-10112 Refuse to start and print txn log information in case of disk corruption
• relates to: CASSANDRA-8271 Remove SSTableDeletingTask
• links to: additional 3.0 patch
• links to: initial-trunk-patch
• links to: Standalone tool dtests