[IMPALA-7142] Ability to use expressions on range partitions

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Frontend
• Epic Color: ghx-label-7
• Assignee: Unassigned
• Reporter: Eli Vingot
• Created: 07/Jun/18 08:56
• Updated: 30/Oct/18 18:29

Description

As a devop I would like to have scripts that create tables for me.
Many times when creating tables the partitioning is dependent on calculated values, mainly time stamps.

Currently we need to alter the script manually for every deployment

 

CREATE TABLE dwh.f_events (
ts BIGINT,
batch_id STRING,
network_id INT,
...
last_updated BIGINT,
PRIMARY KEY (ts,batch_id,network_id)
)
PARTITION BY HASH (network_id) PARTITIONS 8,
RANGE (ts)
(
PARTITION 1524528000000 <= VALUES < 1524614400000
)
STORED AS KUDU

 

 

It would be lovely if we could set the ranges with an expression as below (for 7 days partitions starting today)

CREATE TABLE dwh.f_events (
ts BIGINT,
batch_id STRING,
network_id INT,
...
last_updated BIGINT,
PRIMARY KEY (ts,batch_id,network_id)
)
PARTITION BY HASH (network_id) PARTITIONS 8,
RANGE (ts)
(
PARTITION (unix_timestamp() - (unix_timestamp() % (3600*24) ))*1000) <= VALUES < ((unix_timestamp() - (unix_timestamp() % (3600*24) ))*1000 + (3600000*24*7)))
STORED AS KUDU