Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-1019

Rounding Errors in Floating Point Arithmetic

    Details

    • Type: Improvement
    • Status: Patch Available
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Expression
    • Labels:
      None
    • Environment:

      Ubuntu 14.04
      Hadoop 2.4.1

      Description

      Floating point rounding errors are found when executing some statement on cli. When running math functions with this floating point number, results will be varied based on which JVM "Math" implementation used.

      default> select 0.8-0.1;
      ?minus
      -------------------------------
      0.7000000000000001
      (1 rows, 0.067 sec, 0 B selected)

      With a modified Float8Dotum class, SQL result will be shown like this.

      default> select 0.8-0.1;
      ?minus
      -------------------------------
      0.7
      (1 rows, 0.004 sec, 0 B selected)

        Attachments

          Activity

            People

            • Assignee:
              ykrips Jihun Kang
              Reporter:
              ykrips Jihun Kang
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: