[CASSANDRA-10680] Deal with small compression chunk size better during streaming plan setup

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 2.1.12, 2.2.4, 3.0.1, 3.1
• Component/s: Legacy/Streaming and Messaging
• Severity: Normal
• Assignee: Yuki Morishita
• Reporter: Jeff Jirsa

Authors

Yuki Morishita

Reviewers

Paulo Motta

• Created: 09/Nov/15 20:30
• Updated: 16/Apr/19 09:30
• Resolved: 18/Nov/15 01:33

Description

For clusters using small compression chunk size and terabytes of data, the streaming plan calculations will instantiate hundreds of millions of compressionmetadata$chunk objects, which will create unreasonable amounts of heap pressure. Rather than instantiating all of those at once, streaming should instantiate only as many as needed for a single file per table at a time.

Issue Links

• duplicates: CASSANDRA-10449 OOM on bootstrap after long GC pause