[SPARK-36173] [CORE] Support getting CPU number in TaskContext

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.1.2
• Fix Version/s: 3.3.0
• Component/s: Spark Core
• Assignee: Xiaochang Wu
• Reporter: Xiaochang Wu
• Created: 16/Jul/21 04:10
• Updated: 05/Aug/21 03:18
• Resolved: 05/Aug/21 02:15

Description

In stage-level resource scheduling, the allocated 3rd party resources can be obtained in TaskContext using resources() interface, however there is no API to get how many cpus are allocated for the task. Will add a cpus() interface to TaskContext in addition to resources().

Issue Links

• links to: [Github] Pull Request #33385 (xwu99)
• links to: [Github] Pull Request #33645 (sarutak)
• links to: [Github] Pull Request #33645 (sarutak)