[SPARK-26775] Update Jenkins nodes to support local volumes for K8s integration tests

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.0.0
• Component/s: jenkins, Kubernetes, Spark Core
• Assignee: Shane Knapp
• Reporter: Stavros Kontopoulos
• Created: 29/Jan/19 19:47
• Updated: 17/May/20 18:24
• Resolved: 02/Apr/19 17:20

Description

Current version of Minikube on test machines does not support properly the local persistent volume feature required by this PR: https://github.com/apache/spark/pull/23514.

We get his error:

"spec.local: Forbidden: Local volumes are disabled by feature-gate, metadata.annotations: Required value: Local volume requires node affinity"

This is probably due to this: https://github.com/rancher/rancher/issues/13864 which implies that we need to update to 1.10+ as described in https://kubernetes.io/docs/concepts/storage/volumes/#local. Fabric8io client is already updated in the PR mentioned at the beginning.