Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-10429 Redesign Flink Scheduling, introducing dedicated Scheduler component
  3. FLINK-10432

Introduce bulk/group-aware scheduling

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Later
    • 1.7.0
    • None
    • Runtime / Coordination
    • None

    Description

      In this step, we change our new Scheduler to support reasoning over a bulk of Executions before interacting with the SlotPool instead of requesting slots on a per-execution basis. This gives us the opportunity to group and schedule tasks together as one unit and to apply scheduling algorithms that require a more holistic view on the tasks at hand. For now, this step will probably mainly address streaming / EAGER scheduling. After this step, we expect that we can support the use-case of local recovery.

      Attachments

        Issue Links

          is depended upon by

          Improvement - An improvement or enhancement to an existing feature or task. FLINK-11298 Scheduling job in the unit of concurrent groups

          • Major - Major loss of function.
          • Closed

          Activity

            People

              srichter Stefan Richter
              srichter Stefan Richter
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: