[SPARK-34821] Set up a workflow for developers to run benchmark in their fork

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.2.0
• Fix Version/s: 3.2.0
• Component/s: Project Infra
• Assignee: Hyukjin Kwon
• Reporter: Hyukjin Kwon
• Created: 22/Mar/21 07:52
• Updated: 12/Dec/22 18:11
• Resolved: 03/Apr/21 11:56

Description

Please refer the discussion in https://github.com/apache/spark/pull/31917.
Currently we don't have a pined environment to run the benchmark, and it's difficult to track and follow the performance differences.

It would be great if we have an easy way to get the benchmark results like "Running tests in your forked repository using GitHub Actions" in https://spark.apache.org/developer-tools.html

Issue Links

• is related to: SPARK-35302 Benchmark workflow should create new files for new benchmarks
• is related to: SPARK-34929 MapStatusesSerDeserBenchmark causes a side effect to other benchmarks with tasks being too big (JDK 11)
• is related to: SPARK-34907 Add main class that runs all benchmarks
• is related to: SPARK-34927 Support TPCDSQueryBenchmark in Benchmarks
• links to: [Github] Pull Request #32015 (HyukjinKwon)