[FLINK-21480] Respect external resources from resource requirements

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Done
• Fix Version/s: 1.13.0
• Component/s: Runtime / Coordination
• Labels: pull-request-available
• Assignee: Yangze Guo
• Reporter: Xintong Song
• Created: 24/Feb/21 09:34
• Updated: 22/Mar/21 08:10
• Resolved: 17/Mar/21 10:53

Description

Currently, K8s/Yarn RM reads external resources from configurations and request workers accordingly. That means, all workers will have the same set of external resources.

With the fine-grained resource management allowing TMs with different resources, and applying a strictly matching between requested and registered slot profiles, we need to make sure:

• Allocate workers respecting external resources specified in the slot resource requirements.
• Registered / offered slots should contain the same set of external resources as requested.

Issue Links

• fixes: FLINK-21220 Change the type of cpuCores in ResourceProfile to CPUResource
• links to: GitHub Pull Request #15112