[HIVE-142] Create a metastore check command

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.3.0
• Component/s: Metastore
• Hadoop Flags: Reviewed
• Release Note: HIVE-142. Add a metastore check command. (Johan Oskarsson via zshao)
• Assignee: Johan Oskarsson
• Reporter: Johan Oskarsson
• Created: 09/Dec/08 11:51
• Updated: 02/May/13 02:29
• Resolved: 12/Jan/09 22:30

Description

We need a command to verify that the information in the metastore reflects the data that is on hdfs. For example partitions can be deleted on hdfs but still be in the metastore.

From Joydeep Sen Sarma, see ticket HIVE-126 for the full comment:
for a command line interface - one might want to check the entire database or just a table or even just one partition. other metadata checks will also be added over time (for example - do the file types on disk agree with metadata records, bucketing information etc). So, here's a strawman proposal for a new command:

alter table <DB>[.TABLE [PARTITION-SPEC]] check [TYPE-LIST]

where TYPE by default is 'all' (check for all kinds of errors), but can be specified to a specific type. For example - in this case - we can have a type called 'partitions' (and then over time we can add other types like 'fileformat' etc.). for v1 - we can just drop the type-list altogether.

the check command can produce a list of things that need to be done to fix the format (like adding any directories not in the metastore - but in hdfs - to the metastore). actually performing of such steps would require a user confirmation (y/n).

Issue Links

• is blocked by: HIVE-182 Add database name parameter to getTable and dropTable in Hive.java
• is depended upon by: HIVE-212 Add an option to msck command to continue if a check fails on a table
• is depended upon by: HIVE-213 Msck: Add option to have wild card in table name
• is depended upon by: HIVE-214 Msck: there should be an option to check only tables and not partitions