[HUDI-5608] Support decimals w/ precision > 30 in Column Stats

• Type: Bug
• Status: Open
• Priority: Critical
• Resolution: Unresolved
• Affects Version/s: 0.12.2
• Fix Version/s: 1.1.0
• Component/s: spark
• Story Points: 2
• Epic Link: RFC-27 Multi Modal Indexing
• Assignee: Unassigned
• Reporter: Alexey Kudinkin
• Created: 24/Jan/23 18:38
• Updated: 21/Dec/23 15:28

Description

As reported in: https://github.com/apache/hudi/issues/7732

 

Currently we've limited precision of the supported decimals at 30 assuming that this number is reasonably high to cover 99% of use-cases, but it seems like there's still a demand for even larger Decimals.

The challenge is however to balance the need to support longer Decimals vs storage space we have to provision for each one of them.