[TAJO-744] (Umbrella) ALTER TABLE ADD/DROP PARTITION statement

• Type: New Feature
• Status: In Progress
• Priority: Major
• Resolution: Unresolved
• Component/s: Catalog
• Assignee: JaeHwa Jung
• Reporter: Hyunsik Choi
• Created: 09/Apr/14 08:50
• Updated: 01/Feb/16 01:29

Description

Currently, Tajo does not manage partitioned directly. In Tajo, each partition is just a directory. For each query, a logical planner traverses matched directories in HDFS according to partition predicates.

This approach is not efficient especially in the environment where the number of partitions are very large. It also makes partition management hard.

Tajo should manage partitions directly by using ALTER TABLE ADD/DROP PARTITION statements. A number of partition entries should be stored in the underlying database that catalog uses.

Synopsis of ALTER TABLE ADD/DROP PARTITION

ALTER TABLE table_name [IF NOT EXISTS] ADD COLUMN PARTITION (key1 = 'val2', key2 = 'val2', ...) WITH ('prop_key' = 'prop_val', ...) LOCATION '...';

ALTER TABLE table_name [IF EXISTS] DROP COLUMN PARTITION  (key1 [=|<|<=|>|>=|!=] 'val1', key2 ...,);

Issue Links

• is depended upon by: TAJO-1067 INSERT OVERWRITE INTO should not remove all partitions.
• is related to: TAJO-1253 Catalog API should be queryable
• is related to: TAJO-1176 Implements queryable virtual tables for catalog information

Sub-Tasks

• Allow arbitrary partition path: Open
• Support add multiple partitions in a single ALTER TABLE statement: Open