[SPARK-15565] The default value of spark.sql.warehouse.dir needs to explicitly point to local filesystem

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 2.0.0
• Component/s: SQL
• Target Version/s: 2.0.0
• Assignee: Xiao Li
• Reporter: Yin Huai
• Created: 26/May/16 16:28
• Updated: 13/Oct/16 19:49
• Resolved: 27/May/16 16:54

Description

The default value of spark.sql.warehouse.dir is System.getProperty("user.dir")/warehouse. Since System.getProperty("user.dir") is a local dir, we should explicitly set the scheme to local filesystem.

This should be a one line change (at https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala#L58).

Also see https://issues.apache.org/jira/browse/SPARK-15034?focusedCommentId=15301508&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15301508

Issue Links

• breaks: SPARK-15683 spark sql local FS spark.sql.warehouse.dir throws on YARN
• is related to: SPARK-15034 Use the value of spark.sql.warehouse.dir as the warehouse location instead of using hive.metastore.warehouse.dir
• links to: [Github] Pull Request #13348 (gatorsmile)