[SPARK-35373] Verify checksums of downloaded artifacts in build/mvn

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.4.7, 3.0.2, 3.1.1
• Fix Version/s: 3.0.3, 3.1.2, 3.2.0
• Component/s: Build
• Assignee: Apache Spark
• Reporter: Sean R. Owen
• Created: 11/May/21 16:11
• Updated: 20/May/21 21:00
• Resolved: 13/May/21 14:07

Description

build/mvn is a convenience script that will automatically download Maven (and Scala) if not already present. While it downloads from official ASF mirrors, it does not check the checksum of the artifact, which is available as a .sha512 file from ASF servers.

The risk of a supply chain attack is a bit less theoretical here than usual, because artifacts are downloaded from any of several mirrors worldwide, and injecting a malicious copy of Maven in any one of them might be simpler and less noticeable than injecting it into ASF servers.

(Note, Scala's download site does not seem to provide a checksum. They do all come from Lightbend, at least, not N mirrors. Not much we can do there.)

Issue Links

• is related to: SPARK-35458 ARM CI failed: failed to validate maven sha512
• links to: [Github] Pull Request #32505 (srowen)
• links to: [Github] Pull Request #32591 (dongjoon-hyun)
• links to: [Github] Pull Request #32608 (Yikun)