[YARN-9118] Handle exceptions with parsing user defined GPU devices in GpuDiscoverer

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.3.0, 3.2.1, 3.1.3
• Hadoop Flags: Reviewed
• Assignee: Szilard Nemeth
• Reporter: Szilard Nemeth
• Created: 13/Dec/18 15:02
• Updated: 22/Feb/19 15:05
• Resolved: 22/Feb/19 14:55

Description

getGpusUsableByYarn has the following issues:

• Duplicate GPU device definitions are not denied: This seems to be the biggest issue as it could increase the number of devices on the node if the device ID is defined 2 or more times.
• An empty-string is accepted, it works like the user would not want to use auto-discovery and haven't defined any GPU devices: This will result in an empty device list, but the empty-string check is never explicitly there in the code, so this behavior just coincidental.
• Number validation does not happen on GPU device IDs (separated by commas)

Many testcases are added as the coverage was already very low.

Issue Links

• is depended upon by: YARN-9120 Need to have a way to turn off GPU auto-discovery in GpuDiscoverer