[SPARK-27365] Spark Jenkins supports testing GPU-aware scheduling features

• Type: Story
• Status: Resolved
• Priority: Major
• Resolution: Invalid
• Affects Version/s: 3.0.0
• Component/s: jenkins
• Assignee: Unassigned
• Reporter: Xiangrui Meng
• Created: 04/Apr/19 00:38
• Updated: 12/Dec/22 18:10
• Resolved: 17/May/22 04:01

Description

Upgrade Spark Jenkins to install GPU cards and run GPU integration tests triggered by "GPU" in PRs.

cc: afeng shaneknapp

Issue Links

• blocks: SPARK-27371 Standalone master receives resource info from worker and allocate driver/executor properly
• relates to: SPARK-24615 SPIP: Accelerator-aware task scheduling for Spark

Sub-Tasks

• Get and install GPU cards to Jenkins machines: Resolved
• Design: Spark Jenkins supports GPU integration tests: Resolved