[HIVE-16941] INFORMATION_SCHEMA is too hard to set up and use

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.0.0
• Assignee: Unassigned
• Reporter: Carter Shanklin
• Created: 22/Jun/17 14:16
• Updated: 22/Jun/17 17:55

Description

For anyone else out there struggling, here's what you need:

(1) hive-site changes

In your hive-site, add:

  <property>
    <name>hive.metastore.db.type</name>
    <value>MYSQL</value>
  </property>

Substitute your DB name if it's not MySQL. Note that lower case mysql just won't do, it must be uppercase. Be sure to restart your metastore.

(2) Use the right schematool invocation

You need to use dbType hive with the correct metaDbType + give it the correct JDBC endpoint to HiveServer2.

schematool -metaDbType mysql -dbType hive -initSchema -url jdbc:hive2://localhost:10000/default -driver org.apache.hive.jdbc.HiveDriver

(3) At runtime, add a bunch of JARs

In CLI you can use

add jar /home/vagrant/hivedist/apache-hive-3.0.0-SNAPSHOT-bin/lib/commons-dbcp-1.4.jar;
add jar /home/vagrant/hivedist/apache-hive-3.0.0-SNAPSHOT-bin/lib/commons-dbcp2-2.0.1.jar;
add jar /home/vagrant/hivedist/apache-hive-3.0.0-SNAPSHOT-bin/lib/commons-pool-1.5.4.jar;
add jar /home/vagrant/hivedist/apache-hive-3.0.0-SNAPSHOT-bin/lib/commons-pool2-2.2.jar;
add jar /home/vagrant/hivedist/apache-hive-3.0.0-SNAPSHOT-bin/lib/hive-jdbc-handler-3.0.0-SNAPSHOT.jar;
add jar /usr/share/java/mysql-connector-java.jar;

For HiveServer2 you'll need to deal with AUX JARs or something like that.

–

At a minimum it's important to eliminate Step (3) as it affects end users rather than being a one-time setup pain.

The second biggest pain I had personally was getting schematool to work right, this was largely because of missing validation checks that caused it to do things that had no chance of working. This will be the subject of an additional ticket.

Issue Links

• blocks: HIVE-16946 Information Schema Improvements