Hive
  1. Hive
  2. HIVE-7067

Min() and Max() on Timestamp and Date columns for ORC returns wrong results

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.14.0, 0.13.1
    • Component/s: None
    • Labels:
      None

      Description

      min() and max() of timestamp and date columns of ORC table returns wrong results. The reason for that is when ORC creates object inspectors for date and timestamp it uses JAVA primitive objects as opposed to WRITABLE objects. When get() is performed on java primitive objects, a reference to the underlying object is returned whereas when get() is performed on writable objects, a copy of the underlying object is returned.

      Fix is to change the object inspector creation to return writable objects for timestamp and date.

      1. HIVE-7067.1.patch
        28 kB
        Prasanth Jayachandran
      2. HIVE-7067.2.patch
        31 kB
        Prasanth Jayachandran
      3. HIVE-7067.branch-13.2.patch
        31 kB
        Prasanth Jayachandran

        Issue Links

          Activity

          Hide
          Thejas M Nair added a comment -

          This jira has been fixed as part of 0.13.1 release. If you find further issues, please create a new jira and link it to this one.

          Show
          Thejas M Nair added a comment - This jira has been fixed as part of 0.13.1 release. If you find further issues, please create a new jira and link it to this one.
          Hide
          Sushanth Sowmyan added a comment -

          Committed to 0.13 branch.

          Show
          Sushanth Sowmyan added a comment - Committed to 0.13 branch.
          Hide
          Sushanth Sowmyan added a comment -

          Thanks folks, I'll include this in 0.13.1 RC2.

          Show
          Sushanth Sowmyan added a comment - Thanks folks, I'll include this in 0.13.1 RC2.
          Hide
          Jason Dere added a comment -

          +1 for 0.13.1

          Show
          Jason Dere added a comment - +1 for 0.13.1
          Hide
          Thejas M Nair added a comment -

          +1 for 0.13.1

          Show
          Thejas M Nair added a comment - +1 for 0.13.1
          Hide
          Sushanth Sowmyan added a comment -

          I'm okay with backporting this to 0.13.1, but I want to follow the process I set out for candidates for late inclusion for 0.13.1, and for that, I need two +1s for this. Thejas M Nair / Gunther Hagleitner / Jason Dere , would you please confirm that you're okay with this inclusion request?

          Show
          Sushanth Sowmyan added a comment - I'm okay with backporting this to 0.13.1, but I want to follow the process I set out for candidates for late inclusion for 0.13.1, and for that, I need two +1s for this. Thejas M Nair / Gunther Hagleitner / Jason Dere , would you please confirm that you're okay with this inclusion request?
          Hide
          Prasanth Jayachandran added a comment -

          Committed patch to trunk. Sushanth Sowmyan can you please commit this to 0.13 branch?

          Show
          Prasanth Jayachandran added a comment - Committed patch to trunk. Sushanth Sowmyan can you please commit this to 0.13 branch?
          Hide
          Prasanth Jayachandran added a comment -

          Thanks Thejas for running the tests! Attaching patch that applies cleanly on branch-0.13.

          Show
          Prasanth Jayachandran added a comment - Thanks Thejas for running the tests! Attaching patch that applies cleanly on branch-0.13.
          Hide
          Thejas M Nair added a comment -

          Ran the tests myself and results look good.

          Show
          Thejas M Nair added a comment - Ran the tests myself and results look good.
          Hide
          Hive QA added a comment -

          Overall: -1 no tests executed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12644976/HIVE-7067.2.patch

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/199/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/199/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          

          This message is automatically generated.

          ATTACHMENT ID: 12644976

          Show
          Hive QA added a comment - Overall : -1 no tests executed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12644976/HIVE-7067.2.patch Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/199/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/199/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase This message is automatically generated. ATTACHMENT ID: 12644976
          Hide
          Thejas M Nair added a comment -

          I think we should include this in 0.13.1 release.

          Show
          Thejas M Nair added a comment - I think we should include this in 0.13.1 release.
          Hide
          Prasanth Jayachandran added a comment -

          HIVE QA seems broken. It didn't run any of my previous patches. I will run all the tests locally and will report back if it passes.

          Show
          Prasanth Jayachandran added a comment - HIVE QA seems broken. It didn't run any of my previous patches. I will run all the tests locally and will report back if it passes.
          Hide
          Jason Dere added a comment -

          +1

          Show
          Jason Dere added a comment - +1
          Hide
          Gunther Hagleitner added a comment -

          +1. Will commit if tests pass.

          Show
          Gunther Hagleitner added a comment - +1. Will commit if tests pass.
          Hide
          Prasanth Jayachandran added a comment -

          Addressed Jason Dere's review comments.

          Show
          Prasanth Jayachandran added a comment - Addressed Jason Dere 's review comments.
          Hide
          Prasanth Jayachandran added a comment -

          Attaching RB link

          Show
          Prasanth Jayachandran added a comment - Attaching RB link

            People

            • Assignee:
              Prasanth Jayachandran
              Reporter:
              Prasanth Jayachandran
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development