[HIVE-28196] Preserve column stats when applying UDF upper/lower.

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 4.1.0, 4.0.1
• Labels: hive-4.0.1-merged hive-4.0.1-must performance pull-request-available
• Assignee: Seonggon Namgung
• Reporter: Seonggon Namgung
• Created: 11/Apr/24 04:30
• Updated: 02/Jun/24 11:39
• Resolved: 02/Jun/24 11:39

Description

Current Hive re-estimates column stats (including avgColLen) when it encounters UDF.
In the case of upper and lower, Hive sets avgColLen to hive.stats.max.variable.length.
But these UDFs do not change column stats and the default value(100) is too high for string type key columns, on which upper/lower are usually applied.

This patch keeps input data's avgColLen after applying UDF upper/lower to make a better query plan.

Issue Links

• links to: GitHub Pull Request #5191
• links to: GitHub Pull Request #5263