Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-10046

Hive warehouse dir not set in current directory when not providing hive-site.xml

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.3.1
    • 1.5.3, 1.6.0
    • Spark Core, SQL
    • OS X 10.10.4
      Java 1.7.0_79-b15
      Scala 2.10.5
      Spark 1.3.1

    Description

      When running spark in local environment (for unit-testing purpose) and without providing any `hive-site.xml, databases apart from the default one are created in Hive default hive.metastore.warehouse.dir and not in the current directory (as stated in [Spark docs](http://spark.apache.org/docs/latest/sql-programming-guide.html#hive-tables)). This code snippet, tested with Spark 1.3.1 demonstrates the issue: https://github.com/tmnd1991/spark-hive-bug/blob/master/src/main/scala/Main.scala You cane see that the exception is thrown when executing the CREATE DATABASE STATEMENT, stating that is `Unable to create database path file:/user/hive/warehouse/abc.db, failed to create database abc)` where is `/user/hive/warehouse/abc.db`is not the current directory as stated in the docs.

      Attachments

        Activity

          People

            xwu0226 Xin Wu
            tmnd91 Antonio Murgia
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: