Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-10429 Redesign Flink Scheduling, introducing dedicated Scheduler component
  3. FLINK-14439

RestartPipelinedRegionStrategy leverage tracked partition availability for better failover experience in DefaultScheduler

Attach filesAttach ScreenshotVotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      In current region failover when using DefaultScheduler, most of the input result partition states are unknown. Even though the failure cause is a PartitionException, only one unhealthy partition can be identified.

      The may lead to multiple unsuccessful failovers before all the unhealthy but needed partitions are identified and their producers are involved in the failover as well. (unsuccessful failover here means the recovered tasks get failed again soon due to some missing input partitions.)

      Using JM side tracked partition states to help the region failover to identify unhealthy(missing) partitions earlier can help with this case.

      To achieve it, I'd propose as follows:
      1. Change FailoverStrategy.Factory#create(FailoverTopology) to FailoverStrategy.Factory#create(FailoverTopology, ResultPartitionAvailabilityChecker).
      2. Add schedulerBase#getResultPartitionAvailabilityChecker which returns getExecutionGraph().getResultPartitionAvailabilityChecker()
      3. In DefaultScheduler use the ResultPartitionAvailabilityChecker from SchedulerBase to create the failover strategy from the factory

      It also fails BatchFineGrainedRecoveryITCase due to unexpected failover counts. This is because the legacy scheduler already has similar optimization in FLINK-13055.

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            zhuzh Zhu Zhu
            zhuzh Zhu Zhu
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 20m
                20m

                Slack

                  Issue deployment