Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-17617

Remainder(%) expression.eval returns incorrect result

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.6.3, 2.0.1, 2.1.0
    • Component/s: SQL
    • Labels:

      Description

      Problem

      Remainder(%) expression returns incorrect result when using expression.eval to calculate the result. expression.eval is called in case like constant folding.

      scala> -5083676433652386516D  % 10
      res19: Double = -6.0
      
      // Wrong answer with eval!!!
      scala> Seq("-5083676433652386516D").toDF.select($"value" % 10).show
      |(value % 10)|
      +------------+
      |         0.0|
      +------------+
      
      // Triggers codegen, will  not do constant folding
      scala> sc.makeRDD(Seq("-5083676433652386516D")).toDF.select($"value" % 10).show
      +------------+
      |(value % 10)|
      +------------+
      |        -6.0|
      +------------+
      

      Behavior of postgres:

      seanzhong=# select -5083676433652386516.0  % 10;
       ?column? 
      ----------
           -6.0
      (1 row)
      

        Attachments

          Activity

            People

            • Assignee:
              clockfly Sean Zhong
              Reporter:
              clockfly Sean Zhong
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: