Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-31244

Use Minio instead of Ceph in K8S DepsTestsSuite

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0
    • Fix Version/s: 3.0.0
    • Component/s: Kubernetes, Spark Core, Tests
    • Labels:
      None

      Description

      `DepsTestsSuite` is using `ceph` for S3 storage. However, it's not robust on `minikube` version. Also, the image size is almost 1GB.

      ceph/daemon                               v4.0.3-stable-4.0-nautilus-centos-7-x86_64   a6a05ccdf924        6 months ago        852MB
      ceph/daemon                               v4.0.11-stable-4.0-nautilus-centos-7         87f695550d8e        12 hours ago        901MB
      
      $ minikube version
      minikube version: v1.8.2
      
      $ minikube -p minikube docker-env | source
      
      $ docker run -it --rm -e NETWORK_AUTO_DETECT=4 -e RGW_FRONTEND_PORT=8000 -e SREE_PORT=5001 -e CEPH_DEMO_UID=nano -e CEPH_DAEMON=demo ceph/daemon:v4.0.3-stable-4.0-nautilus-centos-7-x86_64 /bin/sh
      2020-03-25 04:26:21  /opt/ceph-container/bin/entrypoint.sh: ERROR- it looks like we have not been able to discover the network settings
      
      $ docker run -it --rm -e NETWORK_AUTO_DETECT=4 -e RGW_FRONTEND_PORT=8000 -e SREE_PORT=5001 -e CEPH_DEMO_UID=nano -e CEPH_DAEMON=demo ceph/daemon:v4.0.11-stable-4.0-nautilus-centos-7 /bin/sh
      2020-03-25 04:20:30  /opt/ceph-container/bin/entrypoint.sh: ERROR- it looks like we have not been able to discover the network settings
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                dongjoon Dongjoon Hyun
                Reporter:
                dongjoon Dongjoon Hyun
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: