[FLINK-31864] Generate unaligned checkpoint type based on persisted data

• Type: Technical Debt
• Status: Open
• Priority: Not a Priority
• Resolution: Unresolved
• Component/s: Runtime / Checkpointing
• Assignee: Unassigned
• Reporter: Rui Fan
• Created: 20/Apr/23 10:32
• Updated: 20/Apr/23 11:49

Description

FLINK-31588 wants to correct the unaligned checkpoint type after aligned barrier timeout to unaligned barrier on PipelinedSubpartition.

Initially, we wanted to try to keep the semantic of that flag in sync with what StreamTask was actually doing. If checkpoint was unaligned, as it arrived unaligned, it should be reported as such, even if that particular subtask didn't persist any data.

However, we found the code is too complicated, and it probably not worth of making the code so much more complicated, so we prefer generate unaligned checkpoint type based on persisted data.

The detailed discussion: https://github.com/apache/flink/pull/22392#discussion_r1165613057

 

cc pnowojski 

Issue Links

• is caused by: FLINK-27251 Timeout aligned to unaligned checkpoint barrier in the output buffers of an upstream subtask
• is related to: FLINK-31588 The unaligned checkpoint type is wrong at subtask level