[SPARK-45786] Inaccurate Decimal multiplication and division results

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 3.2.4, 3.3.3, 3.4.1, 3.5.0, 4.0.0
• Fix Version/s: 3.4.2, 4.0.0, 3.5.1
• Component/s: SQL
• Labels: correctness pull-request-available
• Target Version/s: 3.4.2
• Assignee: Kazuyuki Tanimura
• Reporter: Kazuyuki Tanimura
• Created: 04/Nov/23 00:35
• Updated: 02/Feb/24 17:07
• Resolved: 07/Nov/23 17:07

Description

Decimal multiplication and division results may be inaccurate due to rounding issues.

Multiplication:

scala> sql("select  -14120025096157587712113961295153.858047 * -0.4652").show(truncate=false)
+----------------------------------------------------+                          
|(-14120025096157587712113961295153.858047 * -0.4652)|
+----------------------------------------------------+
|6568635674732509803675414794505.574764              |
+----------------------------------------------------+

The correct answer is

6568635674732509803675414794505.574763

Please note that the last digit is 3 instead of 4 as

 

scala> java.math.BigDecimal("-14120025096157587712113961295153.858047").multiply(java.math.BigDecimal("-0.4652"))
val res21: java.math.BigDecimal = 6568635674732509803675414794505.5747634644

Since the factional part .574763 is followed by 4644, it should not be rounded up.

Division:

scala> sql("select -0.172787979 / 533704665545018957788294905796.5").show(truncate=false)
+-------------------------------------------------+
|(-0.172787979 / 533704665545018957788294905796.5)|
+-------------------------------------------------+
|-3.237521E-31                                    |
+-------------------------------------------------+

The correct answer is

-3.237520E-31

Please note that the last digit is 0 instead of 1 as

 

scala> java.math.BigDecimal("-0.172787979").divide(java.math.BigDecimal("533704665545018957788294905796.5"), 100, java.math.RoundingMode.DOWN)
val res22: java.math.BigDecimal = -3.237520489418037889998826491401059986665344697406144511563561222578738E-31

Since the factional part .237520 is followed by 4894..., it should not be rounded up.

Issue Links

• links to: GitHub Pull Request #43678
• links to: GitHub Pull Request #43705
• links to: GitHub Pull Request #43853