Hive
  1. Hive
  2. HIVE-5655

Hive incorrecly handles divide-by-zero case

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.0, 0.11.0, 0.12.0
    • Fix Version/s: 0.13.0
    • Component/s: Types
    • Labels:
      None
    • Release Note:
      The behaviour needs to be documented.

      Description

      Unlike other databases, Hive currently has only one mode (default mode) regarding error handling, in which NULL value is returned. However, in case of divide-by-zero, Hive demonstrated a different behavior.

      hive> select 5/0 from tmp2 limit 1;
      Total MapReduce jobs = 1
      ...
      Total MapReduce CPU Time Spent: 860 msec
      OK
      Infinity
      

      The correct behaviour should be Hive returning NULL instead in order to be consistent w.r.t error handling. (BTW, the same situation is handled corrected for decimal type.)

      MySQL has server modes control the behaviour. By default, NULL is returned. For instance,

      mysql> select 3/0 from dual;
      +------+
      | 3/0  |
      +------+
      | NULL |
      +------+
      1 row in set (0.00 sec)
      
      1. HIVE-5655.1.patch
        7 kB
        Xuefu Zhang
      2. HIVE-5655.2.patch
        16 kB
        Xuefu Zhang
      3. HIVE-5655.3.patch
        16 kB
        Xuefu Zhang
      4. HIVE-5655.patch
        7 kB
        Xuefu Zhang

        Issue Links

          Activity

          Hide
          Xuefu Zhang added a comment -
          Show
          Xuefu Zhang added a comment - Review Board Link: https://reviews.apache.org/r/14973/
          Hide
          Xuefu Zhang added a comment -

          Patch # fixed a c&p error.

          Show
          Xuefu Zhang added a comment - Patch # fixed a c&p error.
          Hide
          Edward Capriolo added a comment -

          +1 . Xuefu Zhang We(I) recently committed a new system that runs udf tests through the operator chain. Maybe you want to base your junit test on that.

          see ./ql/src/test/org/apache/hadoop/hive/ql/testutil/BaseScalarUdfTest.java

          Show
          Edward Capriolo added a comment - +1 . Xuefu Zhang We(I) recently committed a new system that runs udf tests through the operator chain. Maybe you want to base your junit test on that. see ./ql/src/test/org/apache/hadoop/hive/ql/testutil/BaseScalarUdfTest.java
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12610460/HIVE-5655.1.patch

          ERROR: -1 due to 2 failed/errored test(s), 4484 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_13
          org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_stats3
          

          Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1256/testReport
          Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1256/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests failed with: TestsFailedException: 2 tests failed
          

          This message is automatically generated.

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12610460/HIVE-5655.1.patch ERROR: -1 due to 2 failed/errored test(s), 4484 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_13 org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_stats3 Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1256/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1256/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests failed with: TestsFailedException: 2 tests failed This message is automatically generated.
          Hide
          Xuefu Zhang added a comment -

          Thanks, Edward. I knew about your proposal of new way of testing udfs, but didn't realize that you have completed that already. I will take a look. Thank you for the review and bringing this up.

          Show
          Xuefu Zhang added a comment - Thanks, Edward. I knew about your proposal of new way of testing udfs, but didn't realize that you have completed that already. I will take a look. Thank you for the review and bringing this up.
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12610460/HIVE-5655.1.patch

          ERROR: -1 due to 2 failed/errored test(s), 4456 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_13
          org.apache.hive.hcatalog.listener.TestNotificationListener.testAMQListener
          

          Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1258/testReport
          Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1258/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests failed with: TestsFailedException: 2 tests failed
          

          This message is automatically generated.

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12610460/HIVE-5655.1.patch ERROR: -1 due to 2 failed/errored test(s), 4456 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_13 org.apache.hive.hcatalog.listener.TestNotificationListener.testAMQListener Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1258/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1258/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests failed with: TestsFailedException: 2 tests failed This message is automatically generated.
          Hide
          Xuefu Zhang added a comment -

          As an update, I'm looking into the HBase test case failure, which I originally thought unrelated. The other test failure is obvious. I just need to generate .out file.

          Show
          Xuefu Zhang added a comment - As an update, I'm looking into the HBase test case failure, which I originally thought unrelated. The other test failure is obvious. I just need to generate .out file.
          Hide
          Xuefu Zhang added a comment -

          Patch #2 fix one of the test failures (vectorization_13.q. Other failures seen above seem unrelated to the changes in the patch. Let's see the test result again.

          Show
          Xuefu Zhang added a comment - Patch #2 fix one of the test failures (vectorization_13.q. Other failures seen above seem unrelated to the changes in the patch. Let's see the test result again.
          Hide
          Hive QA added a comment -

          Overall: +1 all checks pass

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12610758/HIVE-5655.2.patch

          SUCCESS: +1 4517 tests passed

          Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/31/testReport
          Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/31/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          

          This message is automatically generated.

          Show
          Hive QA added a comment - Overall : +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12610758/HIVE-5655.2.patch SUCCESS: +1 4517 tests passed Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/31/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/31/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase This message is automatically generated.
          Hide
          Brock Noland added a comment -

          Looks like the RB item has not been updated with the latest patch. Can you upload it?

          Show
          Brock Noland added a comment - Looks like the RB item has not been updated with the latest patch. Can you upload it?
          Hide
          Brock Noland added a comment -

          Cancelling patch until RB item is addressed.

          Show
          Brock Noland added a comment - Cancelling patch until RB item is addressed.
          Hide
          Xuefu Zhang added a comment -

          Patch #3 rebased with latest trunk. Review board is updated.

          Show
          Xuefu Zhang added a comment - Patch #3 rebased with latest trunk. Review board is updated.
          Hide
          Hive QA added a comment -

          Overall: +1 all checks pass

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12612331/HIVE-5655.3.patch

          SUCCESS: +1 4552 tests passed

          Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/149/testReport
          Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/149/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          

          This message is automatically generated.

          ATTACHMENT ID: 12612331

          Show
          Hive QA added a comment - Overall : +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12612331/HIVE-5655.3.patch SUCCESS: +1 4552 tests passed Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/149/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/149/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase This message is automatically generated. ATTACHMENT ID: 12612331
          Hide
          Brock Noland added a comment -

          +1

          Show
          Brock Noland added a comment - +1
          Hide
          Xuefu Zhang added a comment -

          Patch is committed to trunk. Thanks to Edward and Brock for the reviews.

          Show
          Xuefu Zhang added a comment - Patch is committed to trunk. Thanks to Edward and Brock for the reviews.

            People

            • Assignee:
              Xuefu Zhang
              Reporter:
              Xuefu Zhang
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development