[PARQUET-1487] Do not write original type for timezone-agnostic timestamps

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.11.0
• Fix Version/s: 1.11.0, format-2.7.0
• Component/s: parquet-mr
• Labels: pull-request-available
• Assignee: Nándor Kollár
• Reporter: Zoltan Ivanfi
• Created: 07/Jan/19 14:49
• Updated: 09/Jan/19 12:42
• Resolved: 09/Jan/19 12:42

Description

Historically, the TIMESTAMP_MILLIS and TIMESTAMP_MICROS original types used for the INT64 physical type were always UTC-normalized.

The new TIMESTAMP logical type allows both UTC-normalized and timezone-agnostic timestamps and writes the legacy original types for compatibility reasons. However, the latter should only be written for UTC-normalized timestamps, because legacy readers are not prepared to handle timezone-agnostic timestamps correctly and the original type would just be misleading.

Issue Links

• links to: GitHub Pull Request #125
• links to: GitHub Pull Request #585