[HIVE-5814] Add DATE, TIMESTAMP, DECIMAL, CHAR, VARCHAR types support in HCat

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.12.0
• Fix Version/s: 0.13.0
• Component/s: HCatalog
• Assignee: Eugene Koifman
• Reporter: Eugene Koifman
• Created: 13/Nov/13 18:57
• Updated: 19/May/14 18:51
• Resolved: 05/Feb/14 21:04

Description

Hive 0.12 added support for new data types. Pig 0.12 added some as well. HCat should handle these as well. Also note that CHAR was added recently.

Also allow user to specify a parameter in Pig like so HCatStorer('','', '-onOutOfRangeValue Throw') to control what happens when Pig's value is out of range for target Hive column. Valid values for the option are Throw and Null. Throw - make the runtime raise an exception, Null, which is the default, means NULL is written to target column and a message to that effect is emitted to the log. Only 1 message per column/data type is sent to the log.

See attached HCat-Pig Type Mapping Hive 0.13.pdf for exact mappings.

Issue Links

• incorporates: HIVE-6232 allow user to control out-of-range values in HCatStorer
• is related to: HIVE-6370 LazySimpleSerDe doesn't handle Date and Timestamp properly
• relates to: HIVE-6194 HCatSchema is missing type info for deeply nested fields
• relates to: HIVE-5565 Limit Hive decimal type maximum precision and scale to 38

Sub-Tasks

• remove explicit Joda dependency from itests/hcatalog-unit/pom.xml: Open
• allow user to control out-of-range values in HCatStorer: Resolved
• HCatRecordSerDe should be removed: Open
• Document support for new types in HCat: Open