[STORM-3202] Include offset information to spout metrics and remove storm-kafka-monitor

• Type: Improvement
• Status: In Progress
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.0.0
• Assignee: Stig Rohde Døssing
• Reporter: Jungtaek Lim
• Created: 23/Aug/18 14:08
• Updated: 02/Apr/19 07:52

Description

To provide offset information on Kafka spout (old and new), we have storm-kafka-monitor module which is being run by UI (shell). This approach requires UI doing too many things - basically UI process does most of things via interacting with Nimbus - and also running external shell process in UI process per opening topology page doesn't look right.

We could just let Spout include offset information into spout metric, and let UI leverage the information. I have been thinking about this approach but forgot about addressing it while thinking about generalizing the format. Now I think we don't have to put too much effort to generalize format, because Kafka spout is used mainly.

 

Issue Links

• contains: STORM-2600 Improve or replace storm-kafka-monitor