[OAK-2862] CompactionMap#compress() inefficient for large compaction maps

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.3.0, 1.4
• Component/s: segmentmk
• Labels: compaction doc-impacting gc
• Epic Link: SegmentMK revision GC
• Assignee: Michael Dürig
• Reporter: Michael Dürig
• Created: 11/May/15 14:59
• Updated: 16/Feb/21 09:52
• Resolved: 05/Jun/15 15:11

Description

I've seen CompactionMap#compress() take up most of the time spent in compaction. With 40M record ids in the compaction map compressing runs for hours.

I will back this with numbers as soon as I have a better grip on the issue.

Issue Links

• is depended upon by: OAK-3139 SNFE in persisted comapation map when using CLEAN_OLD
• is depended upon by: OAK-3179 Deadlock between persisted compaction map and cleanup
• is depended upon by: OAK-3264 Deadlock between persisted compaction map and cleanup 2
• is depended upon by: OAK-3317 ConcurrentModificationException when running SegmentOverflowExceptionIT
• is related to: OAK-2896 Putting many elements into a map results in many small segments.
• is related to: OAK-2880 NPE in SegmentWriter.writeMap
• is related to: OAK-3123 NPE in RecordIdMap
• relates to: OAK-2713 High memory usage of CompactionMap