[SPARK-35074] spark.jars.xxx configs should be moved to config/package.scala

• Type: Improvement
• Status: Resolved
• Priority: Trivial
• Resolution: Fixed
• Affects Version/s: 3.2.0
• Fix Version/s: 3.2.0
• Component/s: Spark Core
• Assignee: dgd_contributor
• Reporter: Shardul Mahadik
• Created: 14/Apr/21 15:25
• Updated: 07/Jun/21 17:03
• Resolved: 07/Jun/21 14:57

Description

Currently spark.jars.xxx property keys (e.g. spark.jars.ivySettings and spark.jars.packages) are hardcoded in multiple places within Spark code across multiple modules. We should define them in config/package.scala and reference them in all other places.

This came up during reviews of SPARK-34472 at https://github.com/apache/spark/pull/31591#discussion_r584848624

Issue Links

• is related to: SPARK-34472 SparkContext.addJar with an ivy path fails in cluster mode with a custom ivySettings file
• links to: [Github] Pull Request #32746 (dgd-contributor)