[SPARK-24606] Decimals multiplication and division may be null due to the result precision overflow

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 2.2.1
• Component/s: SQL
• Assignee: Unassigned
• Reporter: Yan Jian
• Created: 20/Jun/18 13:10
• Updated: 20/Jun/18 13:25
• Resolved: 20/Jun/18 13:25

Description

Spark performs mul / div on Decimals via Java's BigDecimal, whose scale may be greater than its precision, with 38 precision limit. 

If the result BigDecimal's precision is 38, and its scale is greater than 38 ( 39 e.g. ), the converted decimal (in spark SQL) is in precision of 40 ( = 39 + 1, and > 38 ).

 

Run following SQLs to reproduce this:

select (cast (1.0 as decimal(38,37))) * 1.8;
select (cast (0.00000777776666655555444443333387654321 as decimal(38,37))) / 99;

Issue Links

• duplicates: SPARK-22036 BigDecimal multiplication sometimes returns null