[HIVE-22893] Enhance data size estimation for fields computed by UDFs

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-1
• Component/s: Statistics
• Labels: pull-request-available
• Assignee: Zoltan Haindrich
• Reporter: Zoltan Haindrich
• Created: 14/Feb/20 16:37
• Updated: 27/Feb/24 22:23
• Resolved: 27/Feb/20 16:26

Description

Right now if we have columnstat on a column ; we use that to estimate things about the column; - however if an UDF is executed on a column ; the resulting column is treated as unknown thing and defaults are assumed.

An improvement could be to give wide estimation(s) in case of frequently used udf.

For example; consider substr(c,1,1) ; no matter what the input; the output is at most a 1 long string

Issue Links

• links to: GitHub Pull Request #915