[ZOOKEEPER-4794] Reduce the ZKDatabase#committedLog memory usage

• Type: Improvement
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 3.9.1
• Fix Version/s: 3.10.0, 3.9.2
• Component/s: server
• Labels: pull-request-available
• Assignee: Yan Zhao
• Reporter: Yan Zhao
• Created: 25/Jan/24 05:01
• Updated: 13/Feb/24 10:23
• Resolved: 09/Feb/24 16:39

Description

In ZKDatabase, after a quorum request is committed successfully, the ZKDatabase will wrap the request into a proposal and store it in the committedLog.
The wrap operation: Serialize the request to a byte array and wrap the byte array in the QuorumPacket, so if the request payload size is 1M, the Proposal will occupy 2M memory, which will increase the memory pressure.

The committedLog is used for fast follower synchronization, so we can serialize the request in the synchronization of the processes, no need to serialize the request in advance.

It can reduce half of the memory for committedLog

Issue Links

• links to: GitHub Pull Request #2115