[CASSANDRA-18280] Investigate initial size of GrowableByteArrayDataOutput in RAMIndexOutput

• Type: Task
• Status: Resolved
• Priority: Normal
• Resolution: Won't Fix
• Component/s: Feature/SAI
• Epic Link: Storage Attached Indexes (Phase 2)
• Platform: All
• Assignee: Unassigned
• Reporter: Mike Adamson
• Created: 23/Feb/23 09:36
• Updated: 07/Sep/23 15:56
• Resolved: 07/Sep/23 15:56

Description

The GrowableByteArrayDataOutput in RAMIndexOutput is currently initialized with a size of 128 bytes. There is no explanation as to why this size was chosen.

The GrowableByteArrayDataOutput does not lazily allocate memory but only ever allocates enough for each write operation. This can lead to a lot of fresh allocations and calls to System.arrayCopy.

Since RAMIndexOutput is used to build the on-disk postings in SAI it is likely that the size of the in-memory array is going to grow considerably more that 128 bytes.

We should investigate changing this initial value to something higher and possibly changing the GrowableByteArrayDataOutput class to allocate in blocks rather than write increments.