[FLINK-25809] Introduce test infra for building FLIP-190 tests

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.19.0
• Labels: pull-request-available
• Assignee: Timo Walther
• Reporter: Francesco Guardiani
• Created: 25/Jan/22 15:54
• Updated: 31/Oct/23 21:54
• Resolved: 31/Oct/23 14:43

Description

The FLIP-190 requires to build a new test infra. For this test infra, we want to define test cases and data once, and then for each case we want to execute the following:

• Integration test that roughly does create plan -> execute job -> trigger savepoint -> stop job -> restore plan -> restore savepoint -> execute job -> stop and assert. Plan and savepoint should be commited to git, so running this tests when a plan and savepoint is available will not regenerate plan and savepoint.
• Change detection test to check that for the defined test cases, the plan hasn't been changed. Similar to the existing JsonPlanITCase tests.
• Completeness of tests/Coverage, that is count how many times ExecNodes (including versions) are used in the test cases. Fail if an ExecNode version is never covered.

Other requirements includes to "version" the test cases, that is for each test case we can retain different versions of the plan and savepoint, in order to make sure that after we introduce a new plan change, the old plan still continues to run

Issue Links

• incorporates: FLINK-33421 Implement ExecNode Restore Tests
• links to: GitHub Pull Request #23584
• links to: GitHub Pull Request #23632