[HIVE-10614] schemaTool upgrade from 0.14.0 to 1.3.0 causes failure

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 1.2.0
• Component/s: Metastore
• Assignee: Hari Sankar Sivarama Subramaniyan
• Reporter: Hari Sankar Sivarama Subramaniyan
• Created: 05/May/15 20:32
• Updated: 18/May/15 19:54
• Resolved: 07/May/15 19:09

Description

./schematool -dbType mysql -upgradeSchemaFrom 0.14.0 -verbose

+----------------------------------------------------------------------------------------------------------------+--+
|                                                                                                                |
+----------------------------------------------------------------------------------------------------------------+--+
| < HIVE-7018 Remove Table and Partition tables column LINK_TARGET_ID from Mysql for other DBs do not have it >  |
+----------------------------------------------------------------------------------------------------------------+--+
1 row selected (0.004 seconds)
0: jdbc:mysql://node-1.example.com/hive> DROP PROCEDURE IF EXISTS RM_TLBS_LINKID
No rows affected (0.005 seconds)
0: jdbc:mysql://node-1.example.com/hive> DROP PROCEDURE IF EXISTS RM_PARTITIONS_LINKID
No rows affected (0.006 seconds)
0: jdbc:mysql://node-1.example.com/hive> DROP PROCEDURE IF EXISTS RM_LINKID
No rows affected (0.002 seconds)
0: jdbc:mysql://node-1.example.com/hive> CREATE PROCEDURE RM_TLBS_LINKID() BEGIN IF EXISTS (SELECT * FROM `INFORMATION_SCHEMA`.`COLUMNS` WHERE `TABLE_NAME` = 'TBLS' AND `COLUMN_NAME` = 'LINK_TARGET_ID') THEN ALTER TABLE `TBLS` DROP FOREIGN KEY `TBLS_FK3` ; ALTER TABLE `TBLS` DROP KEY `TBLS_N51` ; ALTER TABLE `TBLS` DROP COLUMN `LINK_TARGET_ID` ; END IF; END
Error: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '' at line 1 (state=42000,code=1064)

Closing: 0: jdbc:mysql://node-1.example.com/hive?createDatabaseIfNotExist=true
org.apache.hadoop.hive.metastore.HiveMetaException: Upgrade FAILED! Metastore state would be inconsistent !!
org.apache.hadoop.hive.metastore.HiveMetaException: Upgrade FAILED! Metastore state would be inconsistent !!
	at org.apache.hive.beeline.HiveSchemaTool.doUpgrade(HiveSchemaTool.java:229)
	at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:468)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.io.IOException: Schema script failed, errorcode 2
	at org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:355)
	at org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:326)
	at org.apache.hive.beeline.HiveSchemaTool.doUpgrade(HiveSchemaTool.java:224)

Looks like HIVE-7018 has introduced stored procedure as part of mysql upgrade script and it is causing issues with schematool upgrade.

Issue Links

• is broken by: HIVE-7018 Table and Partition tables have column LINK_TARGET_ID in Mysql scripts but not others
• is related to: HIVE-10635 Redo HIVE-7018 in a schematool compatible manner
• relates to: HIVE-10659 Beeline command which contains semi-colon as a non-command terminator will fail