Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-10653 Introduce Pluggable Shuffle Service Architecture
  3. FLINK-13010

Refactor the process of SchedulerNG#requestPartitionState

    XMLWordPrintableJSON

Details

    Description

      Currently `requestPartitionState` is mainly used for querying partition state when the consumer receives `PartitionNotFoundException` during requesting partition. Actually we do not have the concept of partition state atm, and ` requestPartitionState` would return the corresponding producer's state as as result, so it exists a contradiction here.

      My suggestion is refactoring the method as `requestPartitionProducerState` and we do not need to pass `IntermediateDataSetID` and `ResultPartitionID` arguments for finding the corresponding execution attempt. We could only pass the `ExecutionAttemptID` in method then the corresponding execution attempt could be easily found from the mapping in `ExecutionGraph`.

      To do so, we could further remove ` IntermediateDataSetID` from `SingleInputGate` and might replace `IntermediateDataSetID` by `InputGateID` in `InputGateDeploymentDescriptor`.

      Attachments

        Activity

          People

            zjwang Zhijiang
            zjwang Zhijiang
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: