[TAJO-1019] Rounding Errors in Floating Point Arithmetic

• Type: Improvement
• Status: Patch Available
• Priority: Minor
• Resolution: Unresolved
• Component/s: Expression

Environment

Ubuntu 14.04
Hadoop 2.4.1

• Assignee: Jihun Kang
• Reporter: Jihun Kang
• Created: 27/Aug/14 08:14
• Updated: 15/Sep/14 14:13

Description

Floating point rounding errors are found when executing some statement on cli. When running math functions with this floating point number, results will be varied based on which JVM "Math" implementation used.

default> select 0.8-0.1;
?minus
-------------------------------
0.7000000000000001
(1 rows, 0.067 sec, 0 B selected)

With a modified Float8Dotum class, SQL result will be shown like this.

default> select 0.8-0.1;
?minus
-------------------------------
0.7
(1 rows, 0.004 sec, 0 B selected)