[SAMZA-25] Add metrics

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.6.0
• Fix Version/s: 0.7.0
• Component/s: container, kafka, kv, metrics, yarn
• Assignee: Chris Riccomini
• Reporter: Chris Riccomini
• Created: 20/Aug/13 16:13
• Updated: 20/Sep/13 21:53
• Resolved: 20/Sep/13 21:53

Description

The latest refactor dropped a bunch of metrics. We need to re-add them.

Classes that need metrics:

• Add metrics to SamzaContainer/SamzaContainerMetrics
• Add metrics to TaskInstance/TaskInstanceMetrics
• Add metrics to TaskStorageManager
• Add metrics to MessageChooser
• Add metrics to DefaultChooser
• Add metrics to SystemConsumers
• Add metrics to SystemProducers
• Add metrics to KafkaSystemProducer
• Add metrics to KafkaSystemConsumer
• Add metrics to BrokerProxy
• Add metrics to CachedStore
• Add metrics to LoggedStore
• Add metrics to KeyValueStorageEngine
• Add metrics to LevelDbKeyValueStore

We definitely need msgs/sec, bytes/sec at both the container and task level. We also need reads/sec writes/sec on the stores. A lot more, as well.

Issue Links

• is depended upon by: SAMZA-24 Support throttling in Samza
• is depended upon by: SAMZA-28 Use gauge instead of counter for bufferedMessageCountMap