[ORC-370] ORC column statistics should not use java.sql.Date

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Duplicate
• Affects Version/s: 1.4.4, 1.5.1
• Assignee: Owen O'Malley
• Reporter: Prasanth Jayachandran
• Created: 30/May/18 19:24
• Updated: 12/Jan/21 03:16
• Resolved: 26/Aug/20 20:40

Description

ORC PPD evaluation for Date type uses java.sql.Date for min/max comparison causing incorrect results.
Date.compareTo uses millis offset which can return incorrect results depending on the timezone.
Running the testcase in HIVE-19726 passed in Los Angeles but failed in Paris as Date.compareTo return 0 for Los Angeles but returned -1 for Paris.

Similar to bloomfilter, min/max evaluation should use DateWritable (as DateWritable.compareTo uses days offset).

Issue Links

• blocks: HIVE-19726 ORC date PPD is broken
• is superceded by: ORC-661 DateColumnStatistics uses Date, which is not timezone agnostic.
• relates to: ORC-495 [C++] BooleanRleDecoder failed to reset remainingbits to zero on seek
• links to: GitHub Pull Request #388
• links to: GitHub Pull Request #540