[HIVE-20707] Automatic partition management

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 4.0.0-alpha-1
• Target Version/s: 4.0.0
• Assignee: Prasanth Jayachandran
• Reporter: Prasanth Jayachandran
• Created: 08/Oct/18 02:59
• Updated: 27/Feb/24 22:24
• Resolved: 29/Oct/18 22:08

Description

In current scenario, to add partitions for external tables to metastore, MSCK REPAIR command has to be executed manually. To avoid this manual step, external tables can be specified a table property based on which a background metastore thread can sync partitions periodically. Tables can also be specified with partition retention period. Any partition whose age exceeds the retention period will be dropped automatically.

Issue Links

• is related to: HIVE-21072 NPE when running partitioned CTAS statements
• is related to: HIVE-25039 Disable discovery.partitions config for external tables by default
• relates to: HIVE-21103 PartitionManagementTask should not modify DN configs to avoid closing persistence manager
• relates to: HIVE-21482 Partition discovery table property is added to non-partitioned external tables
• relates to: HIVE-21582 Prefix msck configs with metastore
• links to: RB