[FLINK-25144] Test Changelog State Backend manually

• Type: Technical Debt
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.15.0
• Fix Version/s: 1.15.0
• Component/s: Runtime / State Backends
• Labels: release-testing
• Assignee: Roman Khachatryan
• Reporter: Roman Khachatryan
• Created: 02/Dec/21 11:34
• Updated: 11/Mar/22 08:11
• Resolved: 11/Mar/22 08:11

Description

This is a testing task for FLINK-21352 / FLIP-158.

Test plan: https://docs.google.com/document/d/10WVFA0BSR0zrRKRjQbB3iEiC7167MBeAwD6swn_eT24/edit?usp=sharing

Issue Links

• Discovered while testing: FLINK-26063 [Changelog] Incorrect key group logged for PQ.poll and remove
• is blocked by: FLINK-25478 Same materialized state handle should not register multi times
• is blocked by: FLINK-24611 Prevent JM from discarding state on checkpoint abortion
• is blocked by: FLINK-23170 Write metadata after materialization
• is blocked by: FLINK-25446 Avoid sanity check on read bytes on DataInputStream#read(byte[])
• is blocked by: FLINK-25261 Changelog not truncated on materialization
• is blocked by: FLINK-25429 Avoid to close output streams twice during uploading changelogs
• is blocked by: FLINK-25465 Correct the logic of materizating wrapped changelog state
• is related to: FLINK-25260 Recovery fails when using changelog+s3+presto
• is related to: FLINK-21352 FLIP-158: Generalized incremental checkpoints
• relates to: FLINK-25479 Changlog materialization with incremental checkpoint cannot work well in local tests
• relates to: FLINK-25524 If enabled changelog, RocksDB incremental checkpoint would always be full
• Testing discovered: FLINK-26231 [Changelog] Incorrect MaterializationID passed to ChangelogStateBackendHandleImpl
• Testing discovered: FLINK-26306 [Changelog] Thundering herd problem with materialization
• Testing discovered: FLINK-26592 [Changelog] Deadlock in FsStateChangelogWriter
• Testing discovered: FLINK-26590 Triggered checkpoints can be delayed by discarding shared state