[IGNITE-22758] In-memory Raft stores logs on heap for too long

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Labels: ignite-3
• Assignee: Unassigned
• Reporter: Roman Puchkovskiy
• Created: 17/Jul/24 10:27
• Updated: 17/Jul/24 10:33

Description

Raft groups for volatile partitions store Raft logs in memory. Currently, it's stored on heap. They are kept until log gets truncated, which does not seem to happen too often. We need to do something to reclaim the memory.

1. We could restrict in-memory budget and make logs spill on disk when the budget is exceeded. To do this, we could switch from the default 'unlimited budget, no spilling' to a different strategy. Maybe, a strategy looking at an overall size of entries kept in memory could be used (needs to be implemented).
2. We could refuse accepting writes if 'too much memory' is used
3. We could truncate logs more often (especially, given that even an idle partition actually constantly receives safe time sync commands)

Issue Links

• is related to: IGNITE-22331 Storage aimem throws "Failed to commit the transaction." on creation of 1000 tables