[FLINK-25492] Changelog with InMemoryStateChangelogStorage cannot work well

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.15.0
• Component/s: Runtime / State Backends
• Labels: pull-request-available
• Assignee: Hangxiang Yu
• Reporter: Hangxiang Yu
• Created: 31/Dec/21 03:46
• Updated: 08/Feb/22 05:50
• Resolved: 07/Feb/22 12:12

Description

while using ChangelogStataBackend with InMemoryStateChangelogStorage, the seq will be increased when append a record. The materialization will materilalize the record into delegated state backend. But after the materialization finished, the persist method still puts the record into InMemoryChangelogStateHandle which is incorrect.
Just as the patch shows, ChangelogDelegateHashMapInMemoryTest will fail because when restore, the medata cannot be found and the record that should't be there is restored firstly.

Issue Links

• blocks: FLINK-23559 Randomize periodic materialisation interval in tests
• blocks: FLINK-25143 Add ITCase for Generalized incremental checkpoints
• relates to: FLINK-25997 FsStateChangelogWriter cannot work well when there is no change appended
• links to: GitHub Pull Request #18555