[HIVE-3764] Support metastore version consistency check

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.8.0, 0.9.0, 0.10.0, 0.11.0
• Fix Version/s: 0.12.0
• Component/s: Metastore
• Assignee: Prasad Suresh Mujumdar Assign to me
• Reporter: Prasad Suresh Mujumdar
• Created: 03/Dec/12 08:56
• Updated: 31/Mar/17 10:02
• Resolved: 21/Sep/13 20:30

Description

Today there's no version/compatibility information stored in hive metastore. Also the datanucleus configuration property to automatically create missing tables is enabled by default. If you happen to start an older or newer hive or don't run the correct upgrade scripts during migration, the metastore would end up corrupted. The autoCreate schema is not always sufficient to upgrade metastore when migrating to newer release. It's not supported with all databases. Besides the migration often involves altering existing table, changing or moving data etc.

Hence it's very useful to have some consistency check to make sure that hive is using correct metastore and for production systems the schema is not automatically by running hive.

Issue Links

• blocks: HIVE-5301 Add a schema tool for offline metastore schema upgrade
• is related to: HIVE-6113 Upgrade DataNucleus [was: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient]
• is related to: HIVE-6862 add DB schema DDL and upgrade 12to13 scripts for MS SQL Server
• is related to: HIVE-5606 Default Derby metastore_db initial creation fails if hive.metastore.schema.verification=true
• is related to: HIVE-1841 datanucleus.fixedDatastore should be true in hive-default.xml
• is related to: HIVE-12436 Default hive.metastore.schema.verification to true
• relates to: HIVE-5427 TestMetastoreVersion.testVersionRestriction fails on hive 0.12