Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-10046

Hive warehouse dir not set in current directory when not providing hive-site.xml

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.3.1
    • Fix Version/s: 1.5.3, 1.6.0
    • Component/s: Spark Core, SQL
    • Labels:
    • Environment:

      OS X 10.10.4
      Java 1.7.0_79-b15
      Scala 2.10.5
      Spark 1.3.1

      Description

      When running spark in local environment (for unit-testing purpose) and without providing any `hive-site.xml, databases apart from the default one are created in Hive default hive.metastore.warehouse.dir and not in the current directory (as stated in [Spark docs](http://spark.apache.org/docs/latest/sql-programming-guide.html#hive-tables)). This code snippet, tested with Spark 1.3.1 demonstrates the issue: https://github.com/tmnd1991/spark-hive-bug/blob/master/src/main/scala/Main.scala You cane see that the exception is thrown when executing the CREATE DATABASE STATEMENT, stating that is `Unable to create database path file:/user/hive/warehouse/abc.db, failed to create database abc)` where is `/user/hive/warehouse/abc.db`is not the current directory as stated in the docs.

        Attachments

          Activity

            People

            • Assignee:
              xwu0226 Xin Wu
              Reporter:
              tmnd91 Antonio Murgia
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: