[HUDI-6567] ExpressionEvaluators numeric types conversion support

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.14.0
• Component/s: flink, metadata
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: Mark Bukhner
• Created: 19/Jul/23 10:15
• Updated: 01/Aug/23 01:27
• Resolved: 01/Aug/23 01:27

Description

Example:

table tbl (a: bigint)

 
data skipping work only if both types are the same. If we use

select * from tbl where a > 10

data skipping will not work, we have to use explicit cast:

select * from tbl where a > cast(10 as bigint)

 
Problem may be solved converting all numeric types to BigDecimal (without information lose):

by 20 lines of source code, without necessity of adding new rule or filter in flink engine (which can support more complex types conversion, such as string -> integer, but i think there is no need in such conversions)

 

Issue Links

• is part of: HUDI-5491 Support partition push down for flink streaming source
• links to: GitHub Pull Request #9230