[CASSANDRA-6441] Explore merging memtables directly with L1

• Type: Improvement
• Status: Open
• Priority: Low
• Resolution: Unresolved
• Fix Version/s: 5.x
• Component/s: Local/Compaction
• Labels: compaction
• Assignee: Unassigned Assign to me
• Reporter: Jonathan Ellis
• Created: 03/Dec/13 20:46
• Updated: 07/Mar/23 10:54

Description

Currently, memtables flush to L0 and are then compacted with L1, so you automatically have 100% write amplification for unique cells right off the bat.

http://dl.acm.org/citation.cfm?id=2213862 suggests splitting the memtable into pieces corresponding to the ranges of the sstables in L1 and turning the flush + compact into a single write – that is, we'd "compact" the data in the L1 sstable with the corresponding data in the memtable.

This would add some complexity around blocking memtable sections until the corresponding L1 piece is no longer involved in its own compaction with L2, and probably a "panic dump" to the old L0 behavior if we run low on memory. But in theory it sounds like a promising optimization.