[SPARK-4176] Support decimals with precision > 18 in Parquet

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.3.0, 1.3.1, 1.4.0
• Fix Version/s: 1.5.0
• Component/s: SQL
• Target Version/s: 1.5.0
• Assignee: Rene Treffer
• Reporter: Matei Alexandru Zaharia
• Shepherd: liancheng
• Created: 31/Oct/14 22:47
• Updated: 07/Aug/15 12:37
• Resolved: 27/Jul/15 15:30

Description

After https://issues.apache.org/jira/browse/SPARK-3929, only decimals with precisions <= 18 (that can be read into a Long) will be readable from Parquet, so we still need more work to support these larger ones.

Issue Links

• is duplicated by: SPARK-4657 Suport storing decimals in Parquet that don't fit in a LONG
• is related to: SPARK-7196 decimal precision lost when loading DataFrame from JDBC
• links to: [Github] Pull Request #6796 (rtreffer)
• links to: [Github] Pull Request #7455 (liancheng)
• links to: [Github] Pull Request #8031 (liancheng)