[CASSANDRA-13290] Optimizing very small repair streams

• Type: Improvement
• Status: Open
• Priority: Normal
• Resolution: Unresolved
• Component/s: Consistency/Repair
• Assignee: Unassigned
• Reporter: Benjamin Roth
• Created: 02/Mar/17 20:05
• Updated: 16/Apr/19 09:30

Description

I often encountered repair scenarios, where a lot of tiny repair streams were created. This results in hundrets, thousands or even ten-thousands super small SSTables (some bytes to some kbytes).
This puts a lot of pressure on compaction and may even lead to a crash due to too many open files - I also encountered this.

What could help to avoid this:
After CASSANDRA-12888 is resolved, a tiny stream (e.g. < 100kb) could be sent through the write path to be buffered by memtables instead of creating an SSTable each.

Without CASSANDRA-12888 this would break incremental repairs.

Issue Links

• depends upon: CASSANDRA-12888 Incremental repairs broken for MVs and CDC