[CASSANDRA-10280] Make DTCS work well with old data

• Type: Sub-task
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 2.1.12, 2.2.4, 3.0.1, 3.1
• Component/s: Local/Compaction
• Labels: dtcs
• Impacts: Docs
• Assignee: Marcus Eriksson
• Reporter: Marcus Eriksson

Authors

Marcus Eriksson

Reviewers

Branimir Lambov

• Created: 07/Sep/15 16:19
• Updated: 06/Nov/19 22:05
• Resolved: 20/Nov/15 10:11

Description

Operational tasks become incredibly expensive if you keep around a long timespan of data with DTCS - with default settings and 1 year of data, the oldest window covers about 180 days. Bootstrapping a node with vnodes with this data layout will force cassandra to compact very many sstables in this window.

We should probably put a cap on how big the biggest windows can get. We could probably default this to something sane based on max_sstable_age (ie, say we can reasonably handle 1000 sstables per node, then we can calculate how big the windows should be to allow that)

Issue Links

• supercedes: CASSANDRA-9130 reduct default dtcs max_sstable_age