[OAK-6538] Investigate cold standby memory consumption

• Type: Task
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.6.1
• Fix Version/s: 1.7.6, 1.8.0
• Component/s: segment-tar
• Labels: cold-standby
• Assignee: Andrei Dulceanu
• Reporter: Andrei Dulceanu
• Created: 09/Aug/17 12:03
• Updated: 04/Oct/19 17:49
• Resolved: 18/Aug/17 16:04

Description

In an investigation from some time ago, 4GB of heap were needed for transferring 1GB blob and 6GB for 2GB blob. This was in part due to using addTestContent [0] in the investigation, which allocates a huge byte[] on the heap.

OAK-5902 introduced chunking for transferring blobs between primary and standby. This way, the memory needed for syncing a big blob should be around the chunk size used. Solving the way test data is created, it should be possible to transfer a big blob (e.g. 2.5 GB) with less memory.

[0] https://github.com/apache/jackrabbit-oak/blob/trunk/oak-segment-tar/src/test/java/org/apache/jackrabbit/oak/segment/standby/DataStoreTestBase.java#L96

Issue Links

• is related to: OAK-6565 GetBlobResponseEncoder should not write all chunks at once
• relates to: OAK-5902 Cold standby should allow syncing of blobs bigger than 2.2 GB