Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-1508

Add support for reading from SparkConf

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.0.1, 1.1.0
    • Component/s: SQL
    • Labels:
      None

      Description

      Right now we have no ability to configure things in Spark SQL. A good start would be passing a SparkConf though the planner such that users could override the number of partitions used during an Exchange.

      Note that while current spark confs are immutable after the context is created, we want some ability to change settings on a per query basis.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                ConcreteVitamin Zongheng Yang
                Reporter:
                marmbrus Michael Armbrust
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: