Details

      Description

      For operation -605044214913338382 * 55269579109718297360

      expected: -33440539101030154945490585226577271520
      but was: -33440539021801992431226247633033321184

      More generally, if you run TestDecimal128.testHighPrecisionDecimal128Multiply many times, you'll get an occasional failure. This is one example of such a failure.

      1. HIVE-6399.01.patch
        3 kB
        Eric Hanson
      2. HIVE-6399.02.patch
        8 kB
        Eric Hanson
      3. HIVE-6399.02.patch
        8 kB
        Eric Hanson
      4. HIVE-6399.3.patch
        8 kB
        Remus Rusanu
      5. HIVE-6399.4.patch
        11 kB
        Remus Rusanu
      6. HIVE-6399.05.patch
        8 kB
        Eric Hanson

        Issue Links

          Activity

          Hide
          Eric Hanson added a comment -

          Attached patch with explicit test for this known bug in testKnownPriorErrors. No fix yet.

          A quick fix would be to use BigDecimal multiply inside Decimal128 multiply. Although this would not perform well, it'd be safe.

          Show
          Eric Hanson added a comment - Attached patch with explicit test for this known bug in testKnownPriorErrors. No fix yet. A quick fix would be to use BigDecimal multiply inside Decimal128 multiply. Although this would not perform well, it'd be safe.
          Hide
          Eric Hanson added a comment -

          Patch with update to Decimal128.multiplyDestructive() to make it use HiveDecimal.multiply internally, plus updated tests.

          Show
          Eric Hanson added a comment - Patch with update to Decimal128.multiplyDestructive() to make it use HiveDecimal.multiply internally, plus updated tests.
          Hide
          Eric Hanson added a comment -
          Show
          Eric Hanson added a comment - Review board entry: https://reviews.apache.org/r/17972/
          Hide
          Eric Hanson added a comment -

          Uploading again to trigger precommit tests.

          Show
          Eric Hanson added a comment - Uploading again to trigger precommit tests.
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12628632/HIVE-6399.02.patch

          ERROR: -1 due to 1 failed/errored test(s), 5089 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_auto_sortmerge_join_16
          

          Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1300/testReport
          Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1300/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 1 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12628632

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12628632/HIVE-6399.02.patch ERROR: -1 due to 1 failed/errored test(s), 5089 tests executed Failed tests: org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_auto_sortmerge_join_16 Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1300/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1300/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed This message is automatically generated. ATTACHMENT ID: 12628632
          Hide
          Remus Rusanu added a comment -

          I have a fix for the actual error, eta 1 day

          Show
          Remus Rusanu added a comment - I have a fix for the actual error, eta 1 day
          Hide
          Remus Rusanu added a comment -

          3.patch fixes the problem. I don't know what was wrong with the original implementation, my algebra is not up to par. In any case, I replaced the multiply code with the BigInteger.multiplyToLen implementation, which is proven and tested. The code is alloctaion free beacause it uses a ThreadLocal array for intermediate scratch.

          Show
          Remus Rusanu added a comment - 3.patch fixes the problem. I don't know what was wrong with the original implementation, my algebra is not up to par. In any case, I replaced the multiply code with the BigInteger.multiplyToLen implementation, which is proven and tested. The code is alloctaion free beacause it uses a ThreadLocal array for intermediate scratch.
          Show
          Remus Rusanu added a comment - https://reviews.apache.org/r/18071/
          Hide
          Remus Rusanu added a comment -

          This is better patch: the multiply algorithm is directly in LittleEndian logic and the patch contains significant more testing for the multiplyArrays4And4To8 cases.

          Show
          Remus Rusanu added a comment - This is better patch: the multiply algorithm is directly in LittleEndian logic and the patch contains significant more testing for the multiplyArrays4And4To8 cases.
          Hide
          Eric Hanson added a comment -

          Remus' patch is technically good. I have a question I'll raise with the PMC though about the comment about using the algorithm from BigInteger.multiplyToLen. For now I'm going to promote my original patch to get it in so we can get the bug failure out of trunk.

          Show
          Eric Hanson added a comment - Remus' patch is technically good. I have a question I'll raise with the PMC though about the comment about using the algorithm from BigInteger.multiplyToLen. For now I'm going to promote my original patch to get it in so we can get the bug failure out of trunk.
          Hide
          Eric Hanson added a comment -

          Promoting patch 02 to first position to get committed, now as 05.

          Show
          Eric Hanson added a comment - Promoting patch 02 to first position to get committed, now as 05.
          Hide
          Jitendra Nath Pandey added a comment -

          +1 for the latest patch.

          Show
          Jitendra Nath Pandey added a comment - +1 for the latest patch.
          Hide
          Hive QA added a comment -

          Overall: +1 all checks pass

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12629126/HIVE-6399.05.patch

          SUCCESS: +1 5120 tests passed

          Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1332/testReport
          Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1332/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          

          This message is automatically generated.

          ATTACHMENT ID: 12629126

          Show
          Hive QA added a comment - Overall : +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12629126/HIVE-6399.05.patch SUCCESS: +1 5120 tests passed Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1332/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1332/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase This message is automatically generated. ATTACHMENT ID: 12629126
          Hide
          Eric Hanson added a comment -

          Committed to trunk

          Show
          Eric Hanson added a comment - Committed to trunk

            People

            • Assignee:
              Eric Hanson
              Reporter:
              Eric Hanson
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development