[HIVE-21346] Refactor Hive DDL

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Hive
• Labels: refactor-ddl
• Assignee: Miklos Gergely
• Reporter: Miklos Gergely
• Created: 28/Feb/19 12:24
• Updated: 21/Jun/19 21:47

Description

Hive DDL has multiple problems:

• DDLTask is a huge class, needs to be cut to smaller pieces
• DDLSemanticAnalyzer is a huge class, needs to be cut to smaller pieces
• DDLWork has a bad structure by having a field for every possible DDL operation
• DDLDesc subclasses are put into one package which has an endless list of classes, should be separated by subjects
• Many DDLDesc subclasses are not immutable

Sub-Tasks

• Break up DDLTask - extract Database related operations: Closed
• Break up DDLTask - extract Table related operations: Closed
• Break up DDLTask - extract View related operations: Closed
• Break up DDLTask - extract Function related operations: Closed
• Break up DDLTask - extract Privilege related operations: Closed
• Break up DDLTask - extract Workload Management related operations: Closed
• Break up DDLTask - extract Process related operations: Closed
• Break up DDLTask - extract Partition related operations: Closed
• Break up DDLTask - extract Column and Constraint related operations: Closed
• Break up DDLTask - extract Storage related operations: Closed
• Break up DDLTask - extract rest of the Alter Table operations: Closed
• Break up DDLTask - extract rest of the operations: Closed
• Break up DDLTask - cleanup: Closed
• Move Function and Macro related DDL operations into the DDL framework: Closed
• Handle each Alter Database types in a separate desc / operation: Closed
• Move Materialized View Update under DDL: Closed
• Clean up Concatenate and Msck DDL commands: Closed
• Clean up CreateTableLikeOperation: Closed
• Clean up DescTableOperation: Closed
• Cut DropTableDesc/Operation to drop table, view and materialized view: Closed
• Clean up TruncateTable operation and desc: Closed
• Clean up ShowCreateTableOperation: Resolved
• Clean up Add Partition: Closed
• Break up DDLSemanticAnalyzer - extract Database related analyzers: Resolved
• Clean up Drop Partition: Closed
• Break up DDLSemanticAnalyzer - extract Function related analyzers: Resolved
• Break up DDLSemanticAnalyzer - extract Privilege related analyzers: Closed
• Break up DDLSemanticAnalyzer - extract Process related analyzers: Resolved
• Break up DDLSemanticAnalyzer - extract Workload Management related analyzers: Resolved
• Break up DDLSemanticAnalyzer - extract View related analyzers: Resolved
• Break up DDLSemanticAnalyzer - extract Other analyzers: Resolved
• Break up DDLSemanticAnalyzer - extract Table creation analyzers: Resolved
• Break up DDLSemanticAnalyzer - extract Table constraints analyzers: Resolved
• Break up DDLSemanticAnalyzer - extract Table column analyzers: Resolved
• Break up DDLSemanticAnalyzer - extract Table partition analyzers: Resolved
• Break up DDLSemanticAnalyzer - extract Table info and lock analyzers: Resolved
• Break up DDLSemanticAnalyzer - extract Table storage analyzers: Resolved
• Break up DDLSemanticAnalyzer - extract the rest of the Alter Table analyzers: Resolved
• Use the same PerfLogger all over Compiler: Resolved
• Extract Create View analyzer from SemanticAnalyzer: Resolved
• Extract Create Materialized View analyzer from SemanticAnalyzer: Open
• Move show specific codes under DDL and cut MetaDataFormatter classes to pieces: Resolved
• Clean up checkstyle violations in ddl: Resolved
• Remove unnecessary parameter from AbstractAlterTableOperation: Resolved