[DRILL-7947] Unable to aggregate or filter nullable decimals in large record sets

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.19.0
• Fix Version/s: 1.20.0
• Component/s: Storage - Parquet

Environment

Docker: master

Sys.Version: 

1.19.0-SNAPSHOT

82c03e44eac3863a10e599d8f2068fc58e986041

• Assignee: Vova Vysotskyi
• Reporter: Nick Stenroos-Dam
• Created: 04/Jun/21 08:37
• Updated: 11/Jun/21 14:21
• Resolved: 11/Jun/21 14:21

Description

When attempting to aggregate or filter an optional decimal column, that have null values stored in Parquet, the exception: "NumberFormatException: Zero length BigInteger" is thrown.

It only seems to happen on large record sets, and not small record sets.

Test Query:

select
 sum(cost), 
 avg(cost), 
 max(cost), 
 min(cost), 
 sum(sum(cost)) over () 
from data.`ManyRows_OptionalFixedLength_Decimal.parquet`
where cost > 0

Test File:
https://projectbiaps-my.sharepoint.com/:u:/g/personal/nisd_project_bi/EaoM317e421OsnvxHn_S9ecBBffDx1LCt3xn93nJ06f0LQ?e=pVI0Of