[SPARK-28465] K8s integration tests fail due to missing ceph-nano image

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.0.0
• Component/s: Kubernetes, Spark Core
• Assignee: Stavros Kontopoulos
• Reporter: Stavros Kontopoulos
• Created: 21/Jul/19 16:27
• Updated: 17/May/20 18:23
• Resolved: 24/Jul/19 21:58

Description

Image added here: https://github.com/lightbend/spark/blob/72c80ee81ca4c3c9569749b54e2db0ec91b128a5/resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/DepsTestsSuite.scala#L66 needs to be updated to the latest as it was removed from dockerhub.

docker pull ceph/daemon:v4.0.0-stable-4.0-master-centos-7-x86_64
Error response from daemon: manifest for ceph/daemon:v4.0.0-stable-4.0-master-centos-7-x86_64 not found

Also we need to apply this fix: https://github.com/ceph/cn/issues/115#issuecomment-497384369

I will create a PR shortly.

Issue Links

• causes: SPARK-30738 Use specific image version in "Launcher client dependencies" test
• links to: GitHub Pull Request #25222