[SPARK-3929] Support for fixed-precision decimal

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.2.0
• Component/s: SQL
• Target Version/s: 1.2.0
• Assignee: Matei Alexandru Zaharia
• Reporter: Matei Alexandru Zaharia
• Created: 13/Oct/14 17:52
• Updated: 02/Nov/14 03:27
• Resolved: 02/Nov/14 03:27

Description

Spark SQL should support fixed-precision decimals, which are available in Hive 0.13 (see https://cwiki.apache.org/confluence/download/attachments/27362075/Hive_Decimal_Precision_Scale_Support.pdf) as well as in new versions of Parquet. This involves adding precision to the decimal type and implementing various rules for math on it (see above).

Sub-Tasks

• Add precision and scale to Spark SQL's Decimal type: Resolved
• Support reading fixed-precision decimals from Parquet: Resolved
• Support reading fixed-precision decimals from Hive 0.13: Resolved
• Optimize decimal type in Spark SQL for those with small precision: Resolved