[SPARK-28679] Spark Yarn ResourceRequestHelper shouldn't lookup setResourceInformation is no resources specified

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.0.0
• Component/s: Spark Core, YARN
• Assignee: Alessandro Bellina
• Reporter: Thomas Graves
• Created: 09/Aug/19 15:17
• Updated: 17/May/20 18:13
• Resolved: 26/Aug/19 19:01

Description

in the Spark Yarn ResourceRequestHelper it uses reflection to lookup setResourceInformation. We should skip that lookup if the resource Map is empty.

https://github.com/apache/spark/blob/master/resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceRequestHelper.scala#L154

 

Issue Links

• links to: GitHub Pull Request #25403