[OAK-4277] Finalise de-duplication caches

• Type: Task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Segment Tar 0.0.8
• Component/s: segment-tar
• Labels: caching compaction gc monitoring
• Epic Link: SegmentMK revision GC
• Assignee: Michael Dürig
• Reporter: Michael Dürig
• Created: 22/Apr/16 14:29
• Updated: 02/Aug/16 12:45
• Resolved: 27/Jul/16 10:44

Description

OAK-3348 "promoted" the record cache to a de-duplication cache, which is heavily relied upon during compaction. Now also node states go through this cache, which can seen as one concern of the former compaction map (the other being equality).
The current implementation of these caches is quite simple and served its purpose for a POC for getting rid of the "back references" (OAK-3348). Before we are ready for a release we need to finalise a couple of things though:

• Implement cache monitoring and management
• Make cache parameters now hard coded configurable
• Implement proper UTs
• Add proper Javadoc
• Fine tune eviction logic and move it into the caches themselves (instead of relying on the client to evict items pro-actively)
• Fine tune caching strategies: For the node state cache the cost of the item is determined just by its position in the tree. We might want to take further things into account (e.g. number of child nodes). Also we might want to implement pinning so e.g. checkpoints would never be evicted.
• Finally we need to decide who should own this cache. It currently lives with the SegmentWriter. However this is IMO not the correct location as during compaction there is dedicated segment writer whose cache need to be shared with the primary's segment writer upon successful completion.

Issue Links

• is related to: OAK-3348 Cross gc sessions might introduce references to pre-compacted segments