[YARN-5517] Add GPU as a resource type for scheduling

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Component/s: scheduler
• Assignee: Unassigned
• Reporter: Jaeboo Jeong
• Created: 12/Aug/16 07:22
• Updated: 18/Jun/18 05:55
• Resolved: 18/Jun/18 05:55

Description

Currently YARN only support scheduling based on memory and cpu.
There is the issue(YARN-3926) which proposed to extend the YARN resource model.
And there is the issue(YARN-4122) to add support for GPU as a resource using docker.

But these issues didn’t release yet so I just added GPU resource type like memory and cpu.
I don’t consider GPU isolation like YARN-4122.

The properties for GPU resource type is similar to cpu core.

mapred-default.xml
mapreduce.map.gpu.cores (default 0)
mapreduce.reduce.gpu.cores (default 0)
yarn.app.mapreduce.am.resource.gpu-cores (default 0)

yarn-default.xml
yarn.scheduler.minimum-allocation-gcores (default 0)
yarn.scheduler.maximum-allocation-gcores (default 8)
yarn.nodemanager.resource.gcores (default 0)

I attached the patch for branch-2.7.1

Issue Links

• supercedes: YARN-4122 Add support for GPU as a resource