Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-17819

Specified database in JDBC URL is ignored when connecting to thriftserver

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.6.2, 2.0.0, 2.0.1
    • Fix Version/s: 2.0.2, 2.1.0
    • Component/s: SQL
    • Labels:
      None

      Description

      Filing this based on a email thread with Reynold Xin. From the docs, the JDBC connection URL to the thriftserver looks like:

      beeline> !connect jdbc:hive2://<host>:<port>/<database>?hive.server2.transport.mode=http;hive.server2.thrift.http.path=<http_endpoint>
      

      However, anything specified in <database> results in being put in default schema. I'm running these with -e commands, but the shell shows the same behavior.

      In 2.0.1, I created a table foo in schema spark_jira:

      [558|18:01:20] ~/Documents/spark/spark$ bin/beeline -u jdbc:hive2://localhost:10006/spark_jira -n hive -e "show tables"
      Connecting to jdbc:hive2://localhost:10006/spark_jira
      16/10/06 18:01:28 INFO jdbc.Utils: Supplied authorities: localhost:10006
      16/10/06 18:01:28 INFO jdbc.Utils: Resolved authority: localhost:10006
      16/10/06 18:01:28 INFO jdbc.HiveConnection: Will try to open client transport with JDBC Uri: jdbc:hive2://localhost:10006/spark_jira
      Connected to: Spark SQL (version 2.0.1)
      Driver: Hive JDBC (version 1.2.1.spark2)
      Transaction isolation: TRANSACTION_REPEATABLE_READ
      +------------+--------------+--+
      | tableName  | isTemporary  |
      +------------+--------------+--+
      +------------+--------------+--+
      No rows selected (0.558 seconds)
      Beeline version 1.2.1.spark2 by Apache Hive
      Closing: 0: jdbc:hive2://localhost:10006/spark_jira
      [559|18:01:30] ~/Documents/spark/spark$ bin/beeline -u jdbc:hive2://localhost:10006/spark_jira -n hive -e "show tables in spark_jira"
      Connecting to jdbc:hive2://localhost:10006/spark_jira
      16/10/06 18:01:34 INFO jdbc.Utils: Supplied authorities: localhost:10006
      16/10/06 18:01:34 INFO jdbc.Utils: Resolved authority: localhost:10006
      16/10/06 18:01:34 INFO jdbc.HiveConnection: Will try to open client transport with JDBC Uri: jdbc:hive2://localhost:10006/spark_jira
      Connected to: Spark SQL (version 2.0.1)
      Driver: Hive JDBC (version 1.2.1.spark2)
      Transaction isolation: TRANSACTION_REPEATABLE_READ
      +------------+--------------+--+
      | tableName  | isTemporary  |
      +------------+--------------+--+
      | foo        | false        |
      +------------+--------------+--+
      1 row selected (0.664 seconds)
      Beeline version 1.2.1.spark2 by Apache Hive
      Closing: 0: jdbc:hive2://localhost:10006/spark_jira
      

      I also see this in Spark 1.6.2:

      [555|18:13:32] ~/Documents/spark/spark16$ bin/beeline -u jdbc:hive2://localhost:10005/spark_jira -n hive -e "show tables"
      Connecting to jdbc:hive2://localhost:10005/spark_jira
      16/10/06 18:13:37 INFO jdbc.Utils: Supplied authorities: localhost:10005
      16/10/06 18:13:37 INFO jdbc.Utils: Resolved authority: localhost:10005
      16/10/06 18:13:37 INFO jdbc.HiveConnection: Will try to open client transport with JDBC Uri: jdbc:hive2://localhost:10005/spark_jira
      Connected to: Spark SQL (version 1.6.2)
      Driver: Hive JDBC (version 1.2.1.spark2)
      Transaction isolation: TRANSACTION_REPEATABLE_READ
      +--------------+--------------+--+
      |  tableName   | isTemporary  |
      +--------------+--------------+--+
      | all_types    | false        |
      | order_items  | false        |
      | orders       | false        |
      | users        | false        |
      +--------------+--------------+--+
      4 rows selected (0.653 seconds)
      Beeline version 1.2.1.spark2 by Apache Hive
      Closing: 0: jdbc:hive2://localhost:10005/spark_jira
      [556|18:13:39] ~/Documents/spark/spark16$ bin/beeline -u jdbc:hive2://localhost:10005/spark_jira -n hive -e "show tables in spark_jira"
      Connecting to jdbc:hive2://localhost:10005/spark_jira
      16/10/06 18:13:45 INFO jdbc.Utils: Supplied authorities: localhost:10005
      16/10/06 18:13:45 INFO jdbc.Utils: Resolved authority: localhost:10005
      16/10/06 18:13:45 INFO jdbc.HiveConnection: Will try to open client transport with JDBC Uri: jdbc:hive2://localhost:10005/spark_jira
      Connected to: Spark SQL (version 1.6.2)
      Driver: Hive JDBC (version 1.2.1.spark2)
      Transaction isolation: TRANSACTION_REPEATABLE_READ
      +------------+--------------+--+
      | tableName  | isTemporary  |
      +------------+--------------+--+
      | foo        | false        |
      +------------+--------------+--+
      1 row selected (0.633 seconds)
      Beeline version 1.2.1.spark2 by Apache Hive
      Closing: 0: jdbc:hive2://localhost:10005/spark_jira
      

      This appeared to work back in Spark 1.4.1:

      [560|18:17:19] ~/Documents/spark/spark14$ bin/beeline -u jdbc:hive2://localhost:11001/imdb -n hive -e "show tables"
      scan complete in 2ms
      Connecting to jdbc:hive2://localhost:11001/imdb
      Connected to: Spark SQL (version 1.4.1)
      Driver: Spark Project Core (version 1.4.1)
      Transaction isolation: TRANSACTION_REPEATABLE_READ
      +------------------+--------------+
      |    tableName     | isTemporary  |
      +------------------+--------------+
      | aka_name         | false        |
      | aka_title        | false        |
      | cast_info        | false        |
      | char_name        | false        |
      | company_name     | false        |
      | complete_cast    | false        |
      | foo              | false        |
      | keyword          | false        |
      | movie_companies  | false        |
      | movie_info       | false        |
      | movie_keyword    | false        |
      | movie_link       | false        |
      | name             | false        |
      | person_info      | false        |
      | title            | false        |
      +------------------+--------------+
      15 rows selected (0.519 seconds)
      Beeline version 1.4.1 by Apache Hive
      Closing: 0: jdbc:hive2://localhost:11001/imdb
      [561|18:17:36] ~/Documents/spark/spark14$ bin/beeline -u jdbc:hive2://localhost:11001/imdb -n hive -e "show tables in imdb"
      scan complete in 1ms
      Connecting to jdbc:hive2://localhost:11001/imdb
      Connected to: Spark SQL (version 1.4.1)
      Driver: Spark Project Core (version 1.4.1)
      Transaction isolation: TRANSACTION_REPEATABLE_READ
      +------------------+--------------+
      |    tableName     | isTemporary  |
      +------------------+--------------+
      | aka_name         | false        |
      | aka_title        | false        |
      | cast_info        | false        |
      | char_name        | false        |
      | company_name     | false        |
      | complete_cast    | false        |
      | foo              | false        |
      | keyword          | false        |
      | movie_companies  | false        |
      | movie_info       | false        |
      | movie_keyword    | false        |
      | movie_link       | false        |
      | name             | false        |
      | person_info      | false        |
      | title            | false        |
      +------------------+--------------+
      15 rows selected (0.509 seconds)
      Beeline version 1.4.1 by Apache Hive
      Closing: 0: jdbc:hive2://localhost:11001/imdb
      [562|18:17:43] ~/Documents/spark/spark14$ bin/beeline -u jdbc:hive2://localhost:11001/imdb -n hive -e "show tables in default"
      scan complete in 2ms
      Connecting to jdbc:hive2://localhost:11001/imdb
      Connected to: Spark SQL (version 1.4.1)
      Driver: Spark Project Core (version 1.4.1)
      Transaction isolation: TRANSACTION_REPEATABLE_READ
      +------------+--------------+
      | tableName  | isTemporary  |
      +------------+--------------+
      | foo        | false        |
      +------------+--------------+
      1 row selected (0.511 seconds)
      Beeline version 1.4.1 by Apache Hive
      Closing: 0: jdbc:hive2://localhost:11001/imdb
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                dongjoon Dongjoon Hyun
                Reporter:
                tnemet Todd Nemet
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: