Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-19976 FLIP-136: Improve interoperability between DataStream and Table API
  3. FLINK-23372

Disable AllVerticesInSameSlotSharingGroupByDefault in DataStream batch mode




      In order to unify the behavior of DataStream API and Table API batch mode, we should disable AllVerticesInSameSlotSharingGroupByDefault also in DataStream API.

      FLINK-20001 reverted setting this flag but without concrete arguments and the following comment: reconsider actually setting this flag in the future

      After a offline chat with zhuzh, we should introduce this again for consistency:

      The goal to assign different regions to different slot sharing groups by default is to reduce waste of resources. In batch jobs, there can be one region which has data dependency on another region. And the resource computation for slots and managed memory will be affected:
        1. If these regions are in the same slot sharing group, the group will require a large slot which can host tasks from both the regions.
        2. In managed memory fraction computing, tasks from both regions will be considered to compete for managed memory, so each task will be assigned with a smaller managed memory fraction (FLIP-53).
      However, those regions will not run at the same time and results in a waste of resources. 
      For streaming jobs, all tasks will run at the same time. So assigning them to the same slot sharing group will not result resource waste.


        Issue Links



              twalthr Timo Walther
              twalthr Timo Walther
              0 Vote for this issue
              3 Start watching this issue