[YARN-6720] Support updating FPGA related constraint node label after FPGA device re-configuration

• Type: Sub-task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: yarn
• Assignee: Unassigned
• Reporter: Zhankun Tang
• Created: 19/Jun/17 03:16
• Updated: 14/Jul/17 02:36

Description

In order to provide a global optimal scheduling for mutable FPGA resource, it seems an easy and direct way to utilize constraint node labels(YARN-3409) instead of extending the global scheduler(YARN-3926) to match both resource count and attributes.

The rough idea is that the AM sets the constraint node label expression to request containers on the nodes whose FPGA devices has the matching IP, and then NM resource handler update the node constraint label if there's FPGA device re-configuration.

Issue Links

• depends upon: YARN-6507 Add support in NodeManager to isolate FPGA devices with CGroups
• depends upon: YARN-3409 Support Node Attribute functionality
• is related to: YARN-3926 [Umbrella] Extend the YARN resource model for easier resource-type management and profiles