[FLINK-26002] Add test coverage for native format job upgrades

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.15.0
• Fix Version/s: 1.15.0
• Component/s: Runtime / Checkpointing
• Labels: pull-request-available
• Assignee: Anton Kalashnikov
• Reporter: Anton Kalashnikov
• Created: 08/Feb/22 08:51
• Updated: 15/Feb/22 19:03
• Resolved: 15/Feb/22 08:54

Description

1. Initialization of job with an operator with states.
2. Do native savepoint/canonical savepoint/aligned checkpoint
   1. Change job shape(new custom operator) and/or record types
   2. One job, with all possible network exchanges (forward, keyBy, rebalance, broadcast, random, rescale), followed by stateful operator.
   3. No need to modify state schema. Just validate in some way that after upgrade state is assigned to correct operators (so the state should be affecting result of the operators/functions)
   4. Some record type that would allow us to validate consistency of the computations
   5. Validating sink, checking for the consistency
   6. Arbitrary job upgrade. Add two new operators. A chained mapping operator and a another keyed proceeded by keyed exchange . Change record type from int to string, without changing the “logical” value of the record (for example change 1 → "1")
   7. Job upgrade w/o changing the graph record type: no need to test for that
3. Restore from savepoint/checkpoint

Issue Links

• links to: GitHub Pull Request #18658