[SPARK-40969] Unable to download spark 3.3.0 tarball after 3.3.1 release in spark-docker

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.1
• Fix Version/s: 3.4.0
• Component/s: Spark Docker
• Assignee: Qian Sun
• Reporter: Qian Sun
• Created: 31/Oct/22 03:06
• Updated: 01/Nov/22 03:24
• Resolved: 01/Nov/22 03:14

Description

Unable to download spark 3.3.0 tarball in spark-docker.

#7 0.229 + wget -nv -O spark.tgz https://dlcdn.apache.org/spark/spark-3.3.0/spark-3.3.0-bin-hadoop3.tgz
#7 1.061 https://dlcdn.apache.org/spark/spark-3.3.0/spark-3.3.0-bin-hadoop3.tgz:
#7 1.061 2022-10-31 02:59:20 ERROR 404: Not Found.
------
executor failed running [/bin/sh -c set -ex;     export SPARK_TMP="$(mktemp -d)";     cd $SPARK_TMP;     wget -nv -O spark.tgz "$SPARK_TGZ_URL";     wget -nv -O spark.tgz.asc "$SPARK_TGZ_ASC_URL";     export GNUPGHOME="$(mktemp -d)";     gpg --keyserver hkps://keys.openpgp.org --recv-key "$GPG_KEY" ||     gpg --keyserver hkps://keyserver.ubuntu.com --recv-keys "$GPG_KEY";     gpg --batch --verify spark.tgz.asc spark.tgz;     gpgconf --kill all;     rm -rf "$GNUPGHOME" spark.tgz.asc;         tar -xf spark.tgz --strip-components=1;     chown -R spark:spark .;     mv jars /opt/spark/;     mv bin /opt/spark/;     mv sbin /opt/spark/;     mv kubernetes/dockerfiles/spark/decom.sh /opt/;     mv examples /opt/spark/;     mv kubernetes/tests /opt/spark/;     mv data /opt/spark/;     mv python/pyspark /opt/spark/python/pyspark/;     mv python/lib /opt/spark/python/lib/;     cd ..;     rm -rf "$SPARK_TMP";]: exit code: 8

And spark 3.3.1 docker is ok

=> [4/9] RUN set -ex;     export SPARK_TMP="$(mktemp -d)";     cd $SPARK_TMP;     wget -nv -O spark.tgz "https://dlcdn.apache.org/spark/spark-3.3.1/spark-3.3.1-bin-hadoop3.tgz";     wget -nv -O spark.tgz.asc "https://downlo  77.8s
 => [5/9] COPY entrypoint.sh /opt/

Issue Links

• links to: GitHub Pull Request #22