Uploaded image for project: 'ORC'
  1. ORC
  2. ORC-370

ORC column statistics should not use java.sql.Date

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Duplicate
    • 1.4.4, 1.5.1
    • None
    • None
    • None

    Description

      ORC PPD evaluation for Date type uses java.sql.Date for min/max comparison causing incorrect results.
      Date.compareTo uses millis offset which can return incorrect results depending on the timezone.
      Running the testcase in HIVE-19726 passed in Los Angeles but failed in Paris as Date.compareTo return 0 for Los Angeles but returned -1 for Paris.

      Similar to bloomfilter, min/max evaluation should use DateWritable (as DateWritable.compareTo uses days offset).

      Attachments

        Issue Links

          Activity

            People

              omalley Owen O'Malley
              prasanth_j Prasanth Jayachandran
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m