[CASSANDRA-6916] Preemptive opening of compaction result

• Type: Bug
• Status: Resolved
• Priority: Low
• Resolution: Fixed
• Fix Version/s: 2.1 beta2
• Labels: performance
• Severity: Low
• Assignee: Benedict Elliott Smith
• Reporter: Benedict Elliott Smith

Authors

Benedict Elliott Smith

Reviewers

Marcus Eriksson

• Created: 24/Mar/14 16:06
• Updated: 16/Apr/19 09:31
• Resolved: 15/May/14 06:19

Description

Related to CASSANDRA-6812, but a little simpler: when compacting, we mess quite badly with the page cache. One thing we can do to mitigate this problem is to use the sstable we're writing before we've finished writing it, and to drop the regions from the old sstables from the page cache as soon as the new sstables have them (even if they're only written to the page cache). This should minimise any page cache churn, as the old sstables must be larger than the new sstable, and since both will be in memory, dropping the old sstables is at least as good as dropping the new.

The approach is quite straight-forward. Every X MB written:

1. grab flushed length of index file;
2. grab second to last index summary record, after excluding those that point to positions after the flushed length;
3. open index file, and check that our last record doesn't occur outside of the flushed length of the data file (pretty unlikely)
4. Open the sstable with the calculated upper bound

Some complications:

1. must keep running copy of compression metadata for reopening with
2. we need to be able to replace an sstable with itself but a different lower bound
3. we need to drop the old page cache only when readers have finished

Issue Links

• breaks: CASSANDRA-8157 Opening results early with leveled compactions broken
• breaks: CASSANDRA-8683 Ensure early reopening has no overlap with replaced files
• breaks: CASSANDRA-9508 Avoid early opening of compaction results when doing anticompaction
• breaks: CASSANDRA-11886 Streaming will miss sections for early opened sstables during compaction
• breaks: CASSANDRA-13752 Corrupted SSTables created in 3.11
• breaks: CASSANDRA-8034 AssertionError when STOP:ing compactions
• breaks: CASSANDRA-8061 tmplink files are not removed
• breaks: CASSANDRA-8248 Possible memory leak
• breaks: CASSANDRA-8535 java.lang.RuntimeException: Failed to rename XXX to YYY
• breaks: CASSANDRA-8832 SSTableRewriter.abort() should be more robust to failure
• breaks: CASSANDRA-9396 Canonical view of compacting SSTables not working as expected
• breaks: CASSANDRA-10357 mmap file boundary selection is broken for some large files
• breaks: CASSANDRA-10534 CompressionInfo not being fsynced on close
• breaks: CASSANDRA-11468 Reading from early opened, and compressed, sstable throws CorruptSSTableException
• breaks: CASSANDRA-8766 SSTableRewriter opens all sstables as early before completing the compaction
• breaks: CASSANDRA-8750 Ensure SSTableReader.last corresponds exactly with the file end
• depends upon: CASSANDRA-6912 SSTableReader.isReplaced does not allow for safe resource cleanup
• is depended upon by: CASSANDRA-6987 sstablesplit fails in 2.1
• is depended upon by: CASSANDRA-6812 Iterative Memtable->SSTable Replacement
• is related to: CASSANDRA-6746 Reads have a slow ramp up in speed
• relates to: CASSANDRA-8764 Refactor the way we notify compaction strategies about early opened files
• relates to: CASSANDRA-9699 Make choice of sstable reader types explicit
• relates to: CASSANDRA-7167 o.a.c.db.KeyCacheTest unit test failing intermittently in 2.1
• relates to: CASSANDRA-8750 Ensure SSTableReader.last corresponds exactly with the file end
• supercedes: CASSANDRA-6746 Reads have a slow ramp up in speed