[FLINK-13732] Enhance JobManagerMetricGroup with FLIP-6 architecture

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Component/s: Runtime / Metrics
• Assignee: Unassigned
• Reporter: Biao Liu
• Created: 15/Aug/19 03:58
• Updated: 02/Oct/19 17:47
• Resolved: 15/Aug/19 06:42

Description

There is a requirement from user mailing list [1]. I think it's reasonable enough to support.

The scenario is that when deploying a Flink cluster on Yarn, there might be several JM(RM) s running on the same host. IMO that's quite a general scenario. However we can't distinguish the metrics from different JobManagerMetricGroup, because there is only one variable "hostname" we can use.

I think there are some problems of current implementation of JobManagerMetricGroup. It's still non-FLIP-6 style. We should split the metric group into RM and Dispatcher to match the FLIP-6 architecture. And there should be an identification variable supported, just like tm_id.

CC StephanEwen, till.rohrmann, Zentol

1. http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-metrics-scope-for-YARN-single-job-td29389.html

Issue Links

• duplicates: FLINK-9543 Expose JobMaster IDs to metric system
• duplicates: FLINK-9741 Register JVM metrics for each JM separately