Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-14106

Make SlotManager pluggable

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.9.0
    • Fix Version/s: 1.13.0
    • Component/s: Runtime / Coordination
    • Labels:
      None

      Description

      As we are enabling fine grained resource management in 1.10, we can have various resource scheduling strategies. Such strategies generally should make the following three decisions.

      • When to launch new / release existing TMs? (How many TMs)
      • What and how many resources should TMs be started with?
      • How to allocate between slot requests and TM resources?

      We may want to make above decisions differently in different scenarios (active/reactive mode, perjob/session mode, etc.). Therefore, we propose to make the scheduling strategies pluggable.

      We propose to make the following changes:

      • Make SlotManager an interface, and implements it differently for different strategies strategies.
      • Modify ResourceManager-SlotManager interfaces to cover all the three decisions mentioned above in SlotManager. In particular, SlotManager needs to allocate TM resources instead of slot resources from ResourceActions.

        Attachments

        Issue Links

        blocks

        New Feature - A new feature of the product, which has yet to be developed. FLINK-14187 FLIP-56 Dynamic Slot Allocation

        • Major - Major loss of function.
        • Open
        is related to

        Bug - A problem which impairs or prevents the functions of the product. FLINK-19324 Map requested/allocated containers with priority on YARN

        • Major - Major loss of function.
        • Closed
        is required by

        Improvement - An improvement or enhancement to an existing feature or task. FLINK-5791 Support an optimal matching based slot manager for flip6 yarn mode

        • Major - Major loss of function.
        • Closed
        relates to

        Sub-task - The sub-task of the issue FLINK-17061 Unset process/flink memory size from configuration once dynamic worker resource is activated.

        • Major - Major loss of function.
        • Closed
        links to

        Web Link Proposed Changes

          Activity
          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

            • Assignee:
              xintongsong Xintong Song
              Reporter:
              xintongsong Xintong Song

              Dates

              • Created:
                Updated:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 50m
                50m

                  Issue deployment