[SEDONA-208] Use Spark RuntimeConfig in SedonaConf

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.3.1
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: Martin Andersson
• Created: 02/Dec/22 16:12
• Updated: 10/Dec/22 22:30
• Resolved: 10/Dec/22 22:30

Description

Sedona properties can only be set when the spark context is created. That's because properties are read from the spark context instead of SparkSession.conf. It would be nice if properties could be changed at runtime. Just like https://spark.apache.org/docs/latest/configuration.html#runtime-sql-configuration

Then we could do

 

spark = ...
spark.conf.set("sedona.use.index",true)
// execute indexed query
spark.conf.set("sedona.use.index",false)
// execute non indexed query

 

Issue Links

• links to: GitHub Pull Request #722