[SPARK-14212] Add configuration element for --packages option

• Type: New Feature
• Status: Resolved
• Priority: Trivial
• Resolution: Cannot Reproduce
• Affects Version/s: 1.6.1
• Component/s: Documentation, PySpark
• Labels: config starter
• Assignee: Unassigned
• Reporter: Russell Jurney
• Shepherd: Matei Alexandru Zaharia
• Created: 28/Mar/16 20:01
• Updated: 12/Dec/22 18:10
• Resolved: 24/Jan/17 14:30

Description

I use PySpark with the --packages option, for instance to load support for CSV:
pyspark --packages com.databricks:spark-csv_2.10:1.4.0

I would like to not have to set this every time at the command line, so a corresponding element for --packages in the configuration file spark-defaults.conf, would be good to have.