[FLINK-28976] Changelog 1st materialization delayed unneccesarily

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.15.1, 1.16.0
• Fix Version/s: 1.16.0, 1.15.3
• Component/s: Runtime / State Backends
• Labels: pull-request-available
• Assignee: Roman Khachatryan
• Reporter: Roman Khachatryan
• Created: 15/Aug/22 17:27
• Updated: 29/Aug/22 20:09
• Resolved: 26/Aug/22 11:59

Description

In PeriodicMaterializationManager.start(), the 1st materialization is scheduled with a delay: materialization_interval + random_offset

Here, random_offset is added to avoid thundering herd problem.
The next materialization will be scheduled with a delay of only materialization_interval.

That means that the 1st materialization will have to compact up to 2 times more state changes than the subsequent ones.

Which in turn can cause FLINK--26590 or other problems.

Issue Links

• blocks: FLINK-25842 [v2] FLIP-158: Generalized incremental checkpoints
• causes: FLINK-26590 Triggered checkpoints can be delayed by discarding shared state
• links to: GitHub Pull Request #20585