[SPARK-17617] Remainder(%) expression.eval returns incorrect result

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.6.3, 2.0.1, 2.1.0
• Component/s: SQL
• Labels: correctness
• Target Version/s: 1.6.3, 2.0.1, 2.1.0
• Assignee: Sean Zhong
• Reporter: Sean Zhong
• Created: 21/Sep/16 00:02
• Updated: 21/Sep/16 08:58
• Resolved: 21/Sep/16 08:54

Description

Problem

Remainder(%) expression returns incorrect result when using expression.eval to calculate the result. expression.eval is called in case like constant folding.

scala> -5083676433652386516D  % 10
res19: Double = -6.0

// Wrong answer with eval!!!
scala> Seq("-5083676433652386516D").toDF.select($"value" % 10).show
|(value % 10)|
+------------+
|         0.0|
+------------+

// Triggers codegen, will  not do constant folding
scala> sc.makeRDD(Seq("-5083676433652386516D")).toDF.select($"value" % 10).show
+------------+
|(value % 10)|
+------------+
|        -6.0|
+------------+

Behavior of postgres:

seanzhong=# select -5083676433652386516.0  % 10;
 ?column? 
----------
     -6.0
(1 row)

Issue Links

• links to: [Github] Pull Request #15171 (clockfly)