[OAK-1760] RepositoryUpgrade leads to one large commit with DocumentNodeStore causing OOM

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.20
• Fix Version/s: 1.0
• Component/s: mongomk, upgrade
• Assignee: Jukka Zitting
• Reporter: Chetan Mehrotra
• Created: 23/Apr/14 11:15
• Updated: 04/Jun/14 10:16
• Resolved: 24/Apr/14 11:53

Description

RepositoryUpgrade logic currently creates when migrating content from JR2 to Oak currently creates a one large commit. This causes issue with DocumentNodeStore as all the commit state is maintained in memory in Commit#operations variable which leads to OOM when source repository content is large.

On Analyzing the heap dump it can be seen that o.a.j.o.plugins.document.Commit#operations map contains large number of UpdateOp entries. These entries are created due to the diff being performed in o.a.j.o.plugins.document.DocumentNodeStoreBranch#persist which traverse down the whole tree (source being JR2 tree) and captures the diff as UpdateOp. As commit does not get applied till whole diff is complete the in memory state is proportional to repository size

Issue Links

• relates to: OAK-1768 DocumentNodeBuilder.setChildNode() runs OOM with large tree