[SPARK-27835] Resource Scheduling: change driver config from addresses to resourcesFile

• Type: Story
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.0.0
• Component/s: Spark Core
• Epic Link: GPU-aware Scheduling
• Assignee: Thomas Graves
• Reporter: Thomas Graves
• Created: 24/May/19 14:53
• Updated: 30/May/19 12:55
• Resolved: 30/May/19 12:53

Description

currently we added a driver config spark.driver.resources.<resourceName>.addresses for standalone mode.

We should change that to be consistent with the executor side changes and make it spark.driver.resourcesFile. This makes it more flexible in that we wouldn't have to add more configs for different types of resources, standalone mode would just need to output different json in that file.  It will also make some of the parsing code common between the 2.

Issue Links

• links to: GitHub Pull Request #24730