[FLINK-11776] Refactor to simplify the process of scheduleOrUpdateConsumers

• Type: Sub-task
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 1.9.0
• Component/s: Runtime / Coordination
• Labels: pull-request-available
• Assignee: Zhijiang
• Reporter: Zhijiang
• Created: 28/Feb/19 06:42
• Updated: 27/Apr/19 09:47
• Resolved: 27/Apr/19 09:46

Description

Based on the work of FLINK-11417, the process of Execution#scheduleOrUpdateConsumers do not need to consider the race condition with schedule process, so we can refactor to make it easy to handle.

• The concurrent data structure for cacheing partial input channel descriptor can be replaced by common list.
• If the consumer is in CREATED state, we only need to schedule it and the partition info would be known during deployment.
• If the consumer is in SCHEDULED state, we need do nothing.
• If the consumer is in RUNNING state, we can send partition info immediately.
• If the consumer is in DEPLOYING state, we can cache the partition info in order to send them in batch after consumer switching to RUNNING state.
• PartialInputChannelDeploymentDescriptor is not needed any more, we can cache partition info directly.

This refactoring is also a preparation work for future introducing ShuffleMaster in FLINK-11391.

Issue Links

• links to: GitHub Pull Request #7856