[OAK-2140] Segment Compactor will not compact binaries > 16k

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.0.9, 1.1.3
• Component/s: core, segmentmk
• Labels: compaction gc
• Epic Link: SegmentMK revision GC
• Assignee: Alex Deparvu
• Reporter: Alex Deparvu
• Created: 25/Sep/14 15:21
• Updated: 14/Oct/15 09:56
• Resolved: 09/Nov/14 11:18

Description

The compaction bit rely on the SegmentBlob#clone method in the case a binary is being processed but it looks like the #clone contract is not fully enforced for streams that are qualified as 'long values' (>16k if I read the code correctly).
What happens is the stream is initially persisted as chunks in a ListRecord. When compaction calls #clone it will get back the original list of record ids, which will get referenced from the compacted node state [0], making compaction on large binaries ineffective as the bulk segments will never move from the original location where they were created, unless the reference node gets deleted.

I think the original design was setup to prevent large binaries from being copied over but looking at the size problem we have now it might be a good time to reconsider this approach.

[0] https://github.com/apache/jackrabbit-oak/blob/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/segment/SegmentBlob.java#L75

Issue Links

• is depended upon by: OAK-2192 Concurrent commit during compaction results in mixed segments
• is related to: OAK-2045 Long running JCR session prevent live cleanup in Segment FileStore