[HIVE-22405] Add ColumnVector support for ProlepticCalendar

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: storage-2.7.1, 3.1.3, 4.0.0-alpha-1
• Component/s: storage-api
• Labels: compatibility datetime
• Assignee: László Bodor
• Reporter: Owen O'Malley
• Created: 25/Oct/19 21:29
• Updated: 27/Feb/24 22:24
• Resolved: 23/Nov/19 16:51

Description

Hive recently moved its processing to the proleptic calendar, which has created some issues for users who have dates before 1580 AD.

I'd propose extending the column vectors for times & dates to encode which calendar they are using.

• create DateColumnVector that extends LongColumnVector
• add a method to change calendars to both DateColumnVector and TimestampColumnVector.

  /**
   * Change the calendar to or from proleptic. If the new and old values of the flag are the
   * same, nothing is done.
   * useProleptic - set the flag for the proleptic calendar
   * updateData - change the data to match the new value of the flag.
   */
  void changeCalendar(useProleptic: boolean, updateData: boolean);

  /**
   * Detect whether this data is using the proleptic calendar.
   */
  boolean usingProlepticCalendar();

Issue Links

• Is contained by: HIVE-26751 Bug Fixes and Improvements for 3.2.0 release
• is related to: HIVE-22589 Add storage support for ProlepticCalendar in ORC, Parquet, and Avro
• is related to: HIVE-22840 Race condition in formatters of TimestampColumnVector and DateColumnVector