[FLINK-14405] Align ResourceProfile/ResourceSpec fields with the new TaskExecutor memory setups.

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Done
• Fix Version/s: 1.10.0
• Component/s: Runtime / Task
• Labels: pull-request-available
• Assignee: Xintong Song
• Reporter: Xintong Song
• Created: 16/Oct/19 07:12
• Updated: 12/Nov/19 12:53
• Resolved: 12/Nov/19 12:53

Description

According to FLIP-49, a task can consume the following TaskExecutor resources, which will also be the fields of ResourceProfile.

• CPU
• Task Heap Memory
• Task Off-Heap Memory
• On-Heap Managed Memory
• Off-Heap Managed Memory
• Shuffle Memory
• Extended Resources

All the resource requirements should be specified by the operators, except Shuffle Memory which should be decided by JobMaster according to the topology of the physical graph. Therefore, a ResourceSpec should have the following fields.

• CPU
• Task Heap Memory
• Task Off-Heap Memory
• On-Heap Managed Memory
• Off-Heap Managed Memory
• Extended Resources

Issue Links

• links to: GitHub Pull Request #9910
• links to: GitHub Pull Request #9910