[FLINK-21510] ExecutionGraph metrics collide on restart

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.15.0
• Component/s: Runtime / Coordination
• Labels: reactive
• Assignee: Chesnay Schepler
• Reporter: Chesnay Schepler
• Created: 26/Feb/21 07:48
• Updated: 22/Nov/21 21:39
• Resolved: 22/Nov/21 21:39

Description

The ExecutionGraphBuilder registers several metrics directly on the JobManagerJobMetricGroup, which are never cleaned up.

These include upTime/DownTime/restartingTime as well as various checkpointing metrics (see the CheckpointStatsTracker for details; examples are number of checkpoints, checkpoint sizes etc).

When the AdaptiveScheduler re-creates the EG these will collide with metrics of prior attempts.

Essentially we either need to create a separate metric group that we pass to the EG or refactor the metrics to be based on some mutable EG reference.

Issue Links

• causes: FLINK-21855 Document Metrics Limitations
• depends upon: FLINK-21075 FLIP-160: Adaptive scheduler
• is related to: FLINK-21513 Rethink up-/down-/restartingTime metrics

Sub-Tasks

• Remove CheckpointStatsTracker#getJobCheckpointingConfiguration: Closed
• Reuse CheckpointStatsTracker across rescaling: Closed
• AdaptiveScheduler should inform job status listener of all state changes: Closed
• Remove error cause from JobStatusListener interface: Closed
• Move JobStatus-related metrics out of ExecutionGraph: Closed
• Update documentation: Closed