[OAK-6911] Provide a way to tune inline size while storing binaries

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.40.0
• Component/s: segment-tar
• Labels: performance scalability
• Epic Link: TarMK content scalability
• Assignee: Andrei Dulceanu
• Reporter: Chetan Mehrotra
• Created: 07/Nov/17 04:55
• Updated: 25/Jun/21 06:50
• Resolved: 05/Mar/21 11:12

Description

SegmentNodeStore currently inlines binaries of size less that 16KB (Segment.MEDIUM_LIMIT) even if external BlobStore is configured.

Due to this behaviour quite a bit of segment tar storage consist of blob data. In one setup out of 370 GB segmentstore size 290GB is due to inlined binary. If most of this binary content is moved to BlobStore then it would allow same repository to work better in lesser RAM

So it would be useful if some way is provided to disable this default behaviour and let BlobStore take control of inline size i.e. in presence of BlobStore no inlining is attempted by SegmentWriter.

Issue Links

• relates to: OAK-9382 Add binariesInlineThreshold to FileStoreBuilder.toString()
• links to: GitHub Pull Request #280