[FLINK-15603] Show checkpoint start delay in checkpoint statistics

• Type: Improvement
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 1.11.0
• Component/s: Runtime / Metrics, Runtime / Web Frontend
• Labels: pull-request-available usability
• Assignee: Piotr Nowojski
• Reporter: Stephan Ewen
• Created: 15/Jan/20 13:23
• Updated: 21/Jul/20 07:56
• Resolved: 29/Jan/20 08:01

Description

One of the most important metrics is missing in the checkpoint stats: "start delay" (aka "barrier lag"), meaning the time it between when the checkpoint was triggered and when the barriers arrive at a task.

That time is critical to identify if a checkpoint takes too long because of backpressure or other contention.

You can implicitly calculate this by "end_to_end_time - sync_time - async_time", but it is much more obvious for users that something is up when this number is explicitly shown.

Issue Links

• relates to: FLINK-18656 Start Delay metric is always zero for unaligned checkpoints and source tasks
• links to: GitHub Pull Request #10931
• links to: GitHub Pull Request #10935