[FLINK-32199] MetricStore does not remove metrics of nonexistent parallelism in TaskMetricStore when scale down job parallelism

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.17.0
• Fix Version/s: 1.18.0, 1.16.3, 1.17.2
• Component/s: Runtime / Metrics, Runtime / REST
• Labels: pull-request-available
• Assignee: Junrui Li
• Reporter: Junrui Li
• Created: 26/May/23 05:03
• Updated: 05/Jun/23 07:54
• Resolved: 05/Jun/23 07:54

Description

After FLINK-29615, FLINK will update the subtask metrics store when scaling down parallelism. However, task metrics are added in the form of "subtaskIndex + metric.name" or "subtaskIndex + operatorName + metric.name". Users will be able to find many redundant metrics through JobVertexMetricsHandler, which will be very troublesome for users.

Issue Links

• links to: GitHub Pull Request #22681