Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-13806

SQL round() produces incorrect results for negative values

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.6.0, 1.6.1, 2.0.0
    • 1.5.3, 1.6.2, 2.0.0
    • SQL
    • None

    Description

      Round in catalyst/expressions/mathExpressions.scala appears to be untested with negative values, and it doesn't handle them correctly.

      There are at least two issues here:

      First, in the genCode for FloatType and DoubleType with _scale == 0, round() will not produce the same results as for the BigDecimal.ROUND_HALF_UP strategy used in all other cases. This is because Math.round is used for these _scale == 0 cases. For example, Math.round(-3.5) is -3, while BigDecimal.ROUND_HALF_UP at scale 0 for -3.5 is -4.

      Even after this bug is fixed with something like...

      if (${ce.value} < 0) {
        ${ev.value} = -1 * Math.round(-1 * ${ce.value});
      } else {
        ${ev.value} = Math.round(${ce.value});
      }
      

      ...which will allow an additional test like this to succeed in MathFunctionsSuite.scala:

      checkEvaluation(Round(-3.5D, 0), -4.0D, EmptyRow)
      

      ...there still appears to be a problem on at least the checkEvalutionWithUnsafeProjection path, where failures like this are produced:

      Incorrect evaluation in unsafe mode: round(-3.141592653589793, -6), actual: [0,0], expected: [0,8000000000000000] (ExpressionEvalHelper.scala:145)
      

      Attachments

        Activity

          People

            davies Davies Liu
            markhamstra Mark Hamstra
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: