[HIVE-15206] Add a validation functionality to hiveSchemaTool

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.2.0
• Component/s: Metastore
• Assignee: Aihua Xu
• Reporter: Aihua Xu
• Created: 15/Nov/16 15:12
• Updated: 15/Nov/16 15:12

Description

We have seen issues that the metastore get corrupted and cause the whole hiveserver not running.

Add the support to detect such corruption to hiveSchemaTool. Fixing the issue automatically could be risky, so may still defer to the admin.

Sub-Tasks

• Schematool should detect malformed URIs: Resolved
• Schematool should recognize missing tables in metastore: Resolved
• Schematool provides a way to detect invalid entries in VERSION table: Resolved
• Remove unused 'COLUMNS' table from derby schema: Patch Available
• Keep Schema in consistent state where schemaTool fails or succeeds.: Resolved
• Implement a capability to detect incorrect sequence numbers: Resolved
• Detect the values for incorrect NULL values: Resolved
• Refactoring the validate function of HiveSchemaTool to make the output consistent: Resolved
• Location validation for table should ignore the values for view.: Resolved
• Eliminate conflicting output from schematool's table validator.: Resolved
• Change SchemaTool table validator to test based on the dbType: Resolved
• Some minor improvement on the validation tool: Resolved
• SchemaTool should exit with non-zero exit code when one or more validator's fail.: Resolved
• Improve the validation output to make the output to stderr and stdout more consistent: Closed
• Check if the console message from the hive schema tool needs to print to logging file: Open
• Schema table validator should return a sorted list of missing tables: Closed
• Improve location validator to check for blank paths.: Closed
• Improve table validator's performance against Oracle: Closed
• Change the sort key for the schema tool validator to be <ID>: Closed