[HIVE-10362] Support Type check/conversion in dynamic partition column

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.0, 3.0.0, 4.0.0
• Fix Version/s: 4.0.0-alpha-1
• Component/s: Query Processor, Types
• Labels: backward-incompatible
• Assignee: Karen Coppage
• Reporter: Chaoyu Tang
• Created: 16/Apr/15 13:38
• Updated: 17/Nov/22 11:31
• Resolved: 12/Feb/20 17:41

Description

There are quite a lot of issues associated with the non-noramlized or type-mismatched values for partition column. Hive has many ways to introduce such problematic data. HIVE-10307 mainly provides the support to type check/convert/normalize the partition column value in static partition specification. This JIRA tries to deal with the partition column type in dynamic partition insert. Currently any data can be inserted as a partition column value as long as it is quoted as a string. For example,
create table dynparttypechecknum (key int, value string) partitioned by (part int);
insert into dynparttypechecknum partition (part) select key, value, '00001' from src limit 1;
show partitions dynparttypechecknum;
–
part=00001

The partition column value is non-normalized int 00001. It causes some unnecessary problems such as integer partition column JDO filter pushdown (see HIVE-6052) and others like HIVE-10210.

Issue Links

• is duplicated by: HIVE-10210 Compute partition column stats fails when partition value is zero-leading integer
• is related to: HIVE-10210 Compute partition column stats fails when partition value is zero-leading integer
• is related to: IMPALA-3976 Handle partition-key values with multiple synonymous string representations created in Hive.
• is related to: HIVE-23358 MSCK REPAIR should remove all insignificant zeroes from numeric partition values
• links to: Review board