[IMPALA-10948] Impala shouldn't require DECIMAL scale for Parquet files

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 4.1.0
• Component/s: Backend
• Labels: ramp-up
• Epic Color: ghx-label-11
• Assignee: Gergely Fürnstáhl
• Reporter: Zoltán Borók-Nagy
• Created: 01/Oct/21 13:57
• Updated: 03/Aug/22 09:42
• Resolved: 03/Aug/22 09:42

Description

Impala requires the 'scale' to be set for decimal columns: https://github.com/apache/impala/blob/1a61a8025c87c37921a1bba4c49f754d8bd10bcc/be/src/exec/parquet/parquet-metadata-utils.cc#L332

But it is only an optional field in Parquet's SchemaElement and the docs says that if the scale is unspecified then it should be considered to be 0.

Then there's the new logical type annotation DecimalType, but Impala doesn't use it during scans.