Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-13806

SQL round() produces incorrect results for negative values

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.6.0, 1.6.1, 2.0.0
    • Fix Version/s: 1.5.3, 1.6.2, 2.0.0
    • Component/s: SQL
    • Labels:
      None

      Description

      Round in catalyst/expressions/mathExpressions.scala appears to be untested with negative values, and it doesn't handle them correctly.

      There are at least two issues here:

      First, in the genCode for FloatType and DoubleType with _scale == 0, round() will not produce the same results as for the BigDecimal.ROUND_HALF_UP strategy used in all other cases. This is because Math.round is used for these _scale == 0 cases. For example, Math.round(-3.5) is -3, while BigDecimal.ROUND_HALF_UP at scale 0 for -3.5 is -4.

      Even after this bug is fixed with something like...

      if (${ce.value} < 0) {
        ${ev.value} = -1 * Math.round(-1 * ${ce.value});
      } else {
        ${ev.value} = Math.round(${ce.value});
      }
      

      ...which will allow an additional test like this to succeed in MathFunctionsSuite.scala:

      checkEvaluation(Round(-3.5D, 0), -4.0D, EmptyRow)
      

      ...there still appears to be a problem on at least the checkEvalutionWithUnsafeProjection path, where failures like this are produced:

      Incorrect evaluation in unsafe mode: round(-3.141592653589793, -6), actual: [0,0], expected: [0,8000000000000000] (ExpressionEvalHelper.scala:145)
      

        Attachments

          Activity

            People

            • Assignee:
              davies Davies Liu
              Reporter:
              markhamstra Mark Hamstra
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: