[CASSANDRA-10495] Improve the way we do streaming with vnodes

• Type: Improvement
• Status: Open
• Priority: Normal
• Resolution: Unresolved
• Fix Version/s: 5.x
• Component/s: Legacy/Streaming and Messaging
• Assignee: Unassigned
• Reporter: Marcus Eriksson
• Created: 09/Oct/15 06:27
• Updated: 07/Mar/23 10:54

Description

Streaming with vnodes usually creates a large amount of sstables on the target node - for example if each source node has 100 sstables and we use num_tokens = 256, the bootstrapping (for example) node might get 100*256 sstables

One approach could be to do an on-the-fly compaction on the source node, meaning we would only stream out one sstable per range. Note that we will want the compaction strategy to decide how to combine the sstables, for example LCS will not want to mix sstables from different levels while STCS can probably just combine everything

cc yukim