Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-31200

Docker image build fails with Mirror sync in progress? errors.

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Trivial
    • Resolution: Fixed
    • Affects Version/s: 3.0.0
    • Fix Version/s: 3.0.0
    • Component/s: Kubernetes, Spark Core
    • Labels:
      None

      Description

      Following errors would appear, in spite of trying to switch between various mirrors, while building spark docker image.

      bash-3.2$ bin/docker-image-tool.sh -r scrapcodes -t v3.1.0-f1cc86 build
      Sending build context to Docker daemon  203.4MB
      Step 1/18 : ARG java_image_tag=8-jre-slim
      Step 2/18 : FROM openjdk:${java_image_tag}
       ---> 381b20190cf7
      Step 3/18 : ARG spark_uid=185
       ---> Using cache
       ---> 65c06f86753c
      Step 4/18 : RUN set -ex &&     apt-get update &&     ln -s /lib /lib64 &&     apt install -y bash tini libc6 libpam-modules krb5-user libnss3 procps &&     mkdir -p /opt/spark &&     mkdir -p /opt/spark/examples &&     mkdir -p /opt/spark/work-dir &&     touch /opt/spark/RELEASE &&     rm /bin/sh &&     ln -sv /bin/bash /bin/sh &&     echo "auth required pam_wheel.so use_uid" >> /etc/pam.d/su &&     chgrp root /etc/passwd && chmod ug+rw /etc/passwd &&     rm -rf /var/cache/apt/*
       ---> Running in 96bcbe927d35
      + apt-get update
      Get:1 http://deb.debian.org/debian buster InRelease [122 kB]
      Get:2 http://deb.debian.org/debian buster-updates InRelease [49.3 kB]
      Get:3 http://deb.debian.org/debian buster/main amd64 Packages [7907 kB]
      Err:3 http://deb.debian.org/debian buster/main amd64 Packages
        File has unexpected size (13217 != 7906744). Mirror sync in progress? [IP: 151.101.10.133 80]
        Hashes of expected file:
         - Filesize:7906744 [weak]
         - SHA256:80ed5d1cc1f31a568b77e4fadfd9e01fa4d65e951243fd2ce29eee14d4b532cc
         - MD5Sum:80b6d9c1b6630b2234161e42f4040ab3 [weak]
        Release file created at: Sat, 08 Feb 2020 10:57:10 +0000
      Get:5 http://deb.debian.org/debian buster-updates/main amd64 Packages [7380 B]
      Err:5 http://deb.debian.org/debian buster-updates/main amd64 Packages
        File has unexpected size (13233 != 7380). Mirror sync in progress? [IP: 151.101.10.133 80]
        Hashes of expected file:
         - Filesize:7380 [weak]
         - SHA256:6af9ea081b6a3da33cfaf76a81978517f65d38e45230089a5612e56f2b6b789d
        Release file created at: Fri, 20 Mar 2020 02:28:11 +0000
      Get:4 http://security-cdn.debian.org/debian-security buster/updates InRelease [65.4 kB]
      Get:6 http://security-cdn.debian.org/debian-security buster/updates/main amd64 Packages [183 kB]
      Fetched 419 kB in 1s (327 kB/s)
      Reading package lists...
      E: Failed to fetch http://deb.debian.org/debian/dists/buster/main/binary-amd64/by-hash/SHA256/80ed5d1cc1f31a568b77e4fadfd9e01fa4d65e951243fd2ce29eee14d4b532cc  File has unexpected size (13217 != 7906744). Mirror sync in progress? [IP: 151.101.10.133 80]
         Hashes of expected file:
          - Filesize:7906744 [weak]
          - SHA256:80ed5d1cc1f31a568b77e4fadfd9e01fa4d65e951243fd2ce29eee14d4b532cc
          - MD5Sum:80b6d9c1b6630b2234161e42f4040ab3 [weak]
         Release file created at: Sat, 08 Feb 2020 10:57:10 +0000
      E: Failed to fetch http://deb.debian.org/debian/dists/buster-updates/main/binary-amd64/by-hash/SHA256/6af9ea081b6a3da33cfaf76a81978517f65d38e45230089a5612e56f2b6b789d  File has unexpected size (13233 != 7380). Mirror sync in progress? [IP: 151.101.10.133 80]
         Hashes of expected file:
          - Filesize:7380 [weak]
          - SHA256:6af9ea081b6a3da33cfaf76a81978517f65d38e45230089a5612e56f2b6b789d
         Release file created at: Fri, 20 Mar 2020 02:28:11 +0000
      E: Some index files failed to download. They have been ignored, or old ones used instead.
      The command '/bin/sh -c set -ex &&     apt-get update &&     ln -s /lib /lib64 &&     apt install -y bash tini libc6 libpam-modules krb5-user libnss3 procps &&     mkdir -p /opt/spark &&     mkdir -p /opt/spark/examples &&     mkdir -p /opt/spark/work-dir &&     touch /opt/spark/RELEASE &&     rm /bin/sh &&     ln -sv /bin/bash /bin/sh &&     echo "auth required pam_wheel.so use_uid" >> /etc/pam.d/su &&     chgrp root /etc/passwd && chmod ug+rw /etc/passwd &&     rm -rf /var/cache/apt/*' returned a non-zero code: 100
      Failed to build Spark JVM Docker image, please refer to Docker build output for details.
      
      

      Soon, I found that, it was due to my ISP (Since we are working from home now !) trying to intercept http traffic. So it may not happen to others. But, just in case, if some is hit by such errors. Changing mirrors to https from http helped resolve it.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                prashant Prashant Sharma
                Reporter:
                prashant Prashant Sharma
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: