[SAMZA-428] Investigate: how to tune down caching in the KeyValueStore implementations

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: kv
• Assignee: Unassigned
• Reporter: Chinmay Soman
• Created: 01/Oct/14 22:34
• Updated: 23/Oct/14 00:58

Description

Currently, we have a 'CachedStore' layer on top of the KeyValueStore implementation that we use. This might lead to double caching:
i) Once at the CachedStore layer
ii) Possibly cached again in the specific K-V store that we use (for eg: RocksDB / BDB)

We need the CachedStore layer so that the writes to LoggedStore (if configured) are done in an efficient manner.

We can then potentially do some config tuning for the K-V store to reduce its memory footprint and simply write to disk.

Issue Links

• relates to: SAMZA-424 Add a Cache state API to the Samza container