[SPARK-1508] Add support for reading from SparkConf

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.0.1, 1.1.0
• Component/s: SQL
• Assignee: Zongheng Yang
• Reporter: Michael Armbrust
• Created: 15/Apr/14 22:28
• Updated: 10/Jun/14 07:52
• Resolved: 10/Jun/14 07:52

Description

Right now we have no ability to configure things in Spark SQL. A good start would be passing a SparkConf though the planner such that users could override the number of partitions used during an Exchange.

Note that while current spark confs are immutable after the context is created, we want some ability to change settings on a per query basis.

Issue Links

• relates to: SPARK-1191 Convert configs to use SparkConf
• relates to: SPARK-1704 Support EXPLAIN in Spark SQL