[SPARK-23179] Support option to throw exception if overflow occurs during Decimal arithmetic

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.3.0
• Fix Version/s: 3.0.0
• Component/s: SQL
• Assignee: Marco Gaido
• Reporter: Marco Gaido
• Created: 22/Jan/18 14:31
• Updated: 29/Dec/19 09:23
• Resolved: 27/Jun/19 11:05

Description

SQL ANSI 2011 states that in case of overflow during arithmetic operations, an exception should be thrown. This is what most of the SQL DBs do (eg. SQLServer, DB2). Hive currently returns NULL (as Spark does) but HIVE-18291 is open to be SQL compliant.

I propose to have a config option which allows to decide whether Spark should behave according to SQL standards or in the current way (ie. returning NULL).

Issue Links

• is related to: SPARK-28024 Incorrect numeric values when out of range
• is related to: SPARK-28200 Decimal overflow handling in ExpressionEncoder
• is related to: SPARK-28201 Revisit MakeDecimal behavior on overflow
• relates to: HIVE-18291 An exception should be raised if the result is outside the range of decimal
• relates to: SPARK-22036 BigDecimal multiplication sometimes returns null
• links to: [Github] Pull Request #20350 (mgaido91)
• links to: GitHub Pull Request #20350