[HIVE-22589] Add storage support for ProlepticCalendar in ORC, Parquet, and Avro

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-1
• Component/s: Avro, ORC, Parquet
• Labels: compatibility datetime
• Assignee: Jesus Camacho Rodriguez
• Reporter: Jesus Camacho Rodriguez
• Created: 06/Dec/19 08:59
• Updated: 11/Dec/22 07:25
• Resolved: 19/Feb/20 17:36

Description

Hive recently moved its processing to the proleptic calendar, which has created some issues for users who have dates before 1580 AD.

Hive最近将其处理移到了proleptic日历上，这给日期在公元1580年之前的用户带来了一些问题。

HIVE-22405 extended the column vectors for times & dates to encode which calendar they are using.

HIVE-22405扩展了时间和日期的列向量&，以编码它们所使用的日历。

This issue is to support proleptic calendar in ORC, Parquet, and Avro, when files are written/read by Hive. To preserve compatibility with other engines until they upgrade their readers, files will be written using hybrid calendar by default. Default behavior when files do not contain calendar information in their metadata is configurable.

当文件由Hive写入/读取时，此问题用于支持ORC、Parquet和Avro中的proleptic日历。为了在升级阅读器之前保持与其他引擎的兼容性，默认情况下将使用混合日历写入文件。当文件的元数据中不包含日历信息时，默认行为是可配置的。

Issue Links

• incorporates: HIVE-22667 ORC version upgrade including ORC-578
• is blocked by: ORC-578 IllegalArgumentException: Can't use LongColumnVector to read proleptic Gregorian dates.
• Is contained by: HIVE-26751 Bug Fixes and Improvements for 3.2.0 release
• relates to: HIVE-22405 Add ColumnVector support for ProlepticCalendar