[IMPALA-7313] LOAD DATA does not create partitions on-demand (inconsistent with INSERT)

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: Impala 3.0
• Component/s: Docs, Frontend
• Epic Color: ghx-label-1
• Assignee: Unassigned
• Reporter: Todd Lipcon
• Created: 17/Jul/18 23:32
• Updated: 20/Jul/18 18:07

Description

I noticed this inconsistency while working on IMPALA-7311: if a table is partitioned, new partitions are created on-demand by statements like:

INSERT INTO t PARTITION (...) ...

However, the following SQL does not create a partition on demand:

LOAD DATA INPATH '...path...' INTO TABLE t PARTITION (...);

Instead, it fails with "Partition spec does not exist: (...)"

This seems inconsistent and makes the LOAD DATA statement less useful. It seems this should at least be called out in the documentation for LOAD DATA if it's intentional, and perhaps fixed if it's not intentional (perhaps LOAD DATA predates automatic partition creation support?)

Issue Links

• is related to: IMPALA-7330 Make the table metadata refresh after "LOAD" commands incremental