Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-9455

Make SlotManager aware of multi slot TaskManagers

    XMLWordPrintableJSON

    Details

    • Release Note:
      Flink now properly supports TaskExecutors with multiple slots. Consequently, TaskExecutors can now be started with an arbitrary number of slots and it is no longer recommended to start them with a single slot.

      Description

      The SlotManager responsible for managing all available slots of a Flink cluster can request to start new TaskManagers if it cannot fulfill a slot request. The started TaskManager can be started with multiple slots configured but currently, the SlotManager thinks that it will be started with a single slot. As a consequence, it might issue multiple requests to start new TaskManagers even though a single one would be sufficient to fulfill all pending slot requests.

      In order to avoid requesting unnecessary resources which are freed after the idle timeout, I suggest to make the SlotManager aware of how many slots a TaskManager is started with. That way the SlotManager only needs to request a new TaskManager if all of the previously started slots (potentially not yet registered and, thus, future slots) are being assigned to slot requests.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                trohrmann Till Rohrmann
                Reporter:
                trohrmann Till Rohrmann
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: