[HIVE-21216] Write Parquet INT64 timestamp

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-1
• Component/s: Hive
• Labels: backward-incompatible
• Assignee: Karen Coppage Assign to me
• Reporter: Karen Coppage
• Created: 05/Feb/19 14:59
• Updated: 17/Nov/22 11:31
• Resolved: 21/Feb/20 21:25

Description

This patch enables Hive to start writing int64 timestamps in Parquet.

With Parquet version 1.11, a new timestamp LogicalType with base INT64 and the following metadata is introduced:
boolean isAdjustedToUtc: marks whether the timestamp is converted to UTC (aka Instant semantics) or not (LocalDateTime semantics)
enum TimeUnit (NANOS, MICROS, MILLIS): granularity of timestamp

The timestamp will have LocalDateTime semantics (not converted to UTC).
Timestamps outside of range 1677-09-21 00:12:43.145224192 – 2262-04-11 23:47:16.854775807 cannot be written in nanos and will be recorded as NULL.
Time unit (granularity) will be determined by the user. Default is milliseconds.

Issue Links

• is blocked by: HIVE-21215 Read Parquet INT64 timestamp
• links to: review board