[SPARK-35178] maven autodownload failing

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.4.7, 3.0.2, 3.1.1, 3.2.0
• Fix Version/s: 3.0.3, 3.1.2, 3.2.0
• Component/s: Build
• Assignee: Sean R. Owen
• Reporter: Bruce Robbins
• Created: 21/Apr/21 17:26
• Updated: 22/Apr/21 02:01
• Resolved: 22/Apr/21 02:01

Description

I attempted to build a fresh clone of Spark using mvn (on two different networks) and got this error:

exec: curl --silent --show-error -L https://www.apache.org/dyn/closer.lua?action=download&filename=/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.tar.gz
tar: Unrecognized archive format
tar: Error exit delayed from previous errors.
Using `mvn` from path: /tmp/testmvn/spark-mvn-download/build/apache-maven-3.6.3/bin/mvn
build/mvn: line 126: /tmp/testmvn/spark-mvn-download/build/apache-maven-3.6.3/bin/mvn: No such file or directory

if I change the mirror as below, the issue goes away:

-    local APACHE_MIRROR=${APACHE_MIRROR:-'https://www.apache.org/dyn/closer.lua?action=download&filename='}
+    local APACHE_MIRROR=${APACHE_MIRROR:-'https://https://downloads.apache.org'}

 

Issue Links

• links to: [Github] Pull Request #32277 (srowen)
• links to: [Github] Pull Request #32277 (srowen)
• links to: [Github] Pull Request #32282 (dongjoon-hyun)
• links to: [Github] Pull Request #32282 (dongjoon-hyun)