[SAMZA-467] Add latency metrics to the different KeyValueStore layers in Samza

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.9.0
• Component/s: container, kv
• Assignee: Unassigned
• Reporter: Chinmay Soman
• Created: 17/Nov/14 22:36
• Updated: 17/Nov/15 22:21

Description

Currently we expose throughput related metrics in each of the key-value processing layers : (CachedStore, SerialzedStore, RocksDbKeyValueStore, ...). We should also add metrics like:

• avg[Get/Put]LatencyMs
• p95th[Get/Put]LatencyMs
• p99th[Get/Put]LatencyMs

These numbers are useful to correctly reason about the performance of a stateful stream processor. This will also help in debugging the case of a slowdown.

Issue Links

• relates to: SAMZA-349 add timer in metrics
• relates to: SAMZA-421 Test the performance before and after adding the Timer metric