[SPARK-42188] Force SBT protobuf version to match Maven on branch 3.2 and 3.3

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.1, 3.2.3
• Fix Version/s: 3.2.4, 3.3.2
• Component/s: Build
• Assignee: Steve Vaughan
• Reporter: Steve Vaughan
• Created: 25/Jan/23 14:41
• Updated: 26/Jan/23 02:31
• Resolved: 26/Jan/23 02:31

Description

Update SparkBuild.scala to force SBT use of protobuf-java to match the Maven version.  The Maven dependencyManagement section forces protobuf-java to use 2.5.0, but SBT is using 3.14.0.

Snippet from Maven dependency tree

 

[INFO] +- com.google.crypto.tink:tink:jar:1.6.0:compile
[INFO] |  +- com.google.protobuf:protobuf-java:jar:2.5.0:compile    <--- 2.x
[INFO] |  \- com.google.code.gson:gson:jar:2.8.6:compile

  Snippet from SBT dependency tree

[info]   +-com.google.crypto.tink:tink:1.6.0
[info]   | +-com.google.code.gson:gson:2.8.6
[info]   | +-com.google.protobuf:protobuf-java:3.14.0               <--- 3.x

The fix is updating SparkBuild.scala just like SPARK-11538 did with guava.  In addition we should comment on the need to keep the top-level pom.xml and SparkBuild.scala in sync as was done in SPARK-41247

 

Issue Links

• links to: [Github] Pull Request #39745 (snmvaughan)
• links to: [Github] Pull Request #39745 (snmvaughan)
• links to: [Github] Pull Request #39746 (snmvaughan)
• links to: [Github] Pull Request #39746 (snmvaughan)