Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-16605

Add max limitation to the total number of slots

    XMLWordPrintableJSON

    Details

    • Release Note:
      Hide
      With FLINK-16605, user could define the maximum number of slots through "slotmanager.number-of-slots.max" that the Flink cluster allocates.

      This is meant for limiting the resource consumption for batch workloads. It is not recommended to configure this option for streaming workloads, which may fail if there are not enough slots. Note that this configuration option does not take effect for standalone clusters, where how many slots are allocated is not controlled by Flink.
      Show
      With FLINK-16605 , user could define the maximum number of slots through "slotmanager.number-of-slots.max" that the Flink cluster allocates. This is meant for limiting the resource consumption for batch workloads. It is not recommended to configure this option for streaming workloads, which may fail if there are not enough slots. Note that this configuration option does not take effect for standalone clusters, where how many slots are allocated is not controlled by Flink.

      Description

      As discussed in FLINK-15527 and FLINK-15959, we propose to add the max limit to the total number of slots.

      To be specific:

      • Introduce "cluster.number-of-slots.max" configuration option with default value MAX_INT
      • Make the SlotManager respect the max number of slots, when exceeded, it would not allocate resource anymore.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                karmagyz Yangze Guo
                Reporter:
                karmagyz Yangze Guo
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 10m
                  10m