[FLINK-23372] Disable AllVerticesInSameSlotSharingGroupByDefault in DataStream batch mode

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.14.0
• Component/s: API / DataStream
• Labels: pull-request-available
• Assignee: Timo Walther
• Reporter: Timo Walther
• Created: 13/Jul/21 12:52
• Updated: 14/Dec/21 09:05
• Resolved: 15/Jul/21 13:23

Description

In order to unify the behavior of DataStream API and Table API batch mode, we should disable AllVerticesInSameSlotSharingGroupByDefault also in DataStream API.

FLINK-20001 reverted setting this flag but without concrete arguments and the following comment: reconsider actually setting this flag in the future

After a offline chat with zhuzh, we should introduce this again for consistency:

The goal to assign different regions to different slot sharing groups by default is to reduce waste of resources. In batch jobs, there can be one region which has data dependency on another region. And the resource computation for slots and managed memory will be affected:
  1. If these regions are in the same slot sharing group, the group will require a large slot which can host tasks from both the regions.
  2. In managed memory fraction computing, tasks from both regions will be considered to compete for managed memory, so each task will be assigned with a smaller managed memory fraction (FLIP-53).

However, those regions will not run at the same time and results in a waste of resources. 

For streaming jobs, all tasks will run at the same time. So assigning them to the same slot sharing group will not result resource waste.

Issue Links

• causes: FLINK-23593 Performance regression on 15.07.2021
• links to: GitHub Pull Request #16499