[FLINK-19462] Checkpoint statistics for unfinished task snapshots

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.13.0
• Component/s: Runtime / Checkpointing, Runtime / Metrics
• Labels: pull-request-available usability
• Assignee: Roman Khachatryan
• Reporter: Nico Kruber
• Created: 29/Sep/20 18:22
• Updated: 28/May/21 08:06
• Resolved: 25/Jan/21 16:57

Description

If a checkpoint times out, there are currently no stats on the not-yet-finished tasks in the Web UI, so you have to crawl into (debug?) logs.

It would be nice to have these incomplete stats in there instead so that you know quickly what was going on. I could think of these ways to accomplish this:

• the checkpoint coordinator could ask the TMs for it after failing the checkpoint or
• the TMs could send the stats when they notice that the checkpoint is aborted

Maybe there are more options, but I think, this improvement in general would benefit debugging checkpoints.

Issue Links

• causes: FLINK-21217 Resuming Savepoint (rocks, scale up, rocks timers) end-to-end test
• causes: FLINK-21272 Resuming Savepoint (rocks, scale down, rocks timers) end-to-end test' Fail
• causes: FLINK-21122 Update checkpoint_monitoring.zh.md
• is related to: FLINK-20886 Add the option to get a threaddump on checkpoint timeouts
• links to: GitHub Pull Request #14635
• mentioned in: Page Loading...