[SPARK-38302] Use Java 17 in K8S integration tests when setting spark-tgz

• Type: Sub-task
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.3.0
• Fix Version/s: 3.3.0
• Component/s: Kubernetes, Tests
• Assignee: Qian Sun
• Reporter: Qian Sun
• Created: 23/Feb/22 10:12
• Updated: 02/Mar/22 20:04
• Resolved: 26/Feb/22 23:58

Description

When setting parameters `spark-tgz` during integration tests, the error that `resource-managers/kubernetes/docker/src/main/dockerfiles/spark/Dockerfile.java17` cannot be found occurs. This is due to the default value of `spark.kubernetes.test.dockerFile` being `resource-managers/kubernetes/docker/src/main/dockerfiles/spark/Dockerfile.java17`. When using the tgz, the working directory is `${spark.kubernetes.test.unpackSparkDir}`, and the relative path `resource-managers/kubernetes/docker/src/main/dockerfiles/spark/Dockerfile.java17` is invalid.

Issue Links

• links to: [Github] Pull Request #35627 (dcoliversun)