[FLINK-13441] Add batch sql E2E test which runs with fewer slots than parallelism

• Type: Improvement
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 1.9.0
• Component/s: API / DataSet, Tests
• Labels: pull-request-available
• Assignee: Alex
• Reporter: Till Rohrmann
• Created: 26/Jul/19 15:38
• Updated: 27/Apr/21 18:08
• Resolved: 08/Aug/19 12:27

Description

We should adapt the existing batch E2E test to use the newly introduced ScheduleMode#LAZY_FROM_SOURCES_WITH_BATCH_SLOT_REQUEST and verify that the job runs on a cluster with fewer slots than the job's parallelism. In order to make this work, we need to set the shuffles to be blocking via ExecutionMode#BATCH. As a batch job we should use the DataSetAllroundTestProgram.

 Update: currently, the ScheduleMode#LAZY_FROM_SOURCES_WITH_BATCH_SLOT_REQUEST option is set only by table planner(s) and cannot be set (configured) for general purpose (batch) job. As agreed offline, this ticket would add a new e2e test for batch sql job instead of modifying DataSetAllroundTestProgram.

 

Issue Links

• links to: GitHub Pull Request #9359