Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-15626

Remove legacy scheduler

    XMLWordPrintableJSON

    Details

      Description

      This umbrella ticket is to track the tickets to remove the legacy scheduler and related components.
      So that we can have a much cleaner scheduler framework which significantly simplifies our next development work on job scheduling.

      Here's a rough plan:
      1. remove LegacyScheduler class

      • related tests should also be removed, including AlsoRunWithLegacyScheduler which is used to enable integration tests for legacy scheduler

      2. remove legacy failover strategy AdaptedRestartPipelinedRegionStrategyNG and RestartIndividualStrategy. These implementations and their tests heavily relies on the legacy scheduling logics in ExecutionGraph so removing them can make #3 much easier.

      3. remove legacy scheduling/failover logics in ExecutionGraph

      • note that many tests still relies on these logics(e.g. ExecutionGraph#scheduleForExecution()), we may need to retain and rewrite tests which are still needed. So before removing the legacy logics, it's better to cleanup all the tests relying on the legacy scheduling first, i.e. remove tests that are no needed and rewrite the tests which are still needed.

      4. remove legacy FailoverStrategy and RestartStrategy

      5. Factoring out the scheduling logics in ExecutionGraph to make it a plain data structure. This can be a long term goal though. There can be several pieces and we may need to examine them one by one.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              zhuzh Zhu Zhu
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 0.5h
                0.5h