[YUNIKORN-2170] e2e test setup takes a lot of time

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.5.0
• Component/s: test - e2e
• Labels: pull-request-available
• Target Version: 1.5.0
• Assignee: Kuan Po Tseng
• Reporter: Peter Bacsko
• Created: 21/Nov/23 13:26
• Updated: 20/Mar/24 14:32
• Resolved: 27/Nov/23 15:19

Description

Setting up e2e test takes significant amount of time.

For example: https://github.com/apache/yunikorn-k8shim/actions/runs/6927721705/job/18842345746?pr=730

The whole run took 1h 18min.
e2e tests: 34min.
Downloading Spark: 40min.

Mon, 20 Nov 2023 09:06:35 GMT installing shellcheck v0.9.0
Mon, 20 Nov 2023 09:06:35 GMT installing golangci-lint v1.54.2
Mon, 20 Nov 2023 09:06:36 GMT installing kubectl v1.27.7
Mon, 20 Nov 2023 09:06:36 GMT installing kind v0.20.0
Mon, 20 Nov 2023 09:06:36 GMT installing helm v3.12.1
Mon, 20 Nov 2023 09:06:37 GMT installing spark 3.3.1
Mon, 20 Nov 2023 09:45:48 GMT Cloning into './build/yunikorn-release'...
Mon, 20 Nov 2023 09:45:48 GMT step 3/7: building docker images from latest code

Options:

• Use a different mirror if there's any or set up one
• Take advantage of Github caching: https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows

Issue Links

• links to: GitHub Pull Request #731
• links to: GitHub Pull Request #732