[HIVE-28334] Support queries with function expression in the prepare execute workflow

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.1.0, 4.0.1
• Labels: hive-4.0.1-merged hive-4.0.1-must pull-request-available
• Assignee: Ramesh Kumar Thangarajan
• Reporter: Ramesh Kumar Thangarajan
• Created: 18/Jun/24 00:24
• Updated: 25/Sep/24 04:44
• Resolved: 09/Jul/24 23:28

Description

We see the below error for queries with function expression fail for prepare execute workflow

 java.lang.NullPointerException
    at org.apache.hadoop.hive.ql.stats.StatsUtils.getSizeOfComplexTypes(StatsUtils.java:1203)
    at org.apache.hadoop.hive.ql.stats.StatsUtils.getAvgColLenOf(StatsUtils.java:1187)
    at org.apache.hadoop.hive.ql.stats.StatsUtils.getColStatisticsFromExpression(StatsUtils.java:1670)
    at org.apache.hadoop.hive.ql.stats.StatsUtils.getColStatisticsFromExprMap(StatsUtils.java:1491)
    at org.apache.hadoop.hive.ql.optimizer.stats.annotation.StatsRulesProcFactory$SelectStatsRule.process(StatsRulesProcFactory.java:232)
    at org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
    at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:105)
    at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:89)
    at org.apache.hadoop.hive.ql.lib.LevelOrderWalker.walk(LevelOrderWalker.java:148)
    at org.apache.hadoop.hive.ql.lib.LevelOrderWalker.startWalking(LevelOrderWalker.java:125)
    at org.apache.hadoop.hive.ql.optimizer.stats.annotation.AnnotateWithStatistics.transform(AnnotateWithStatistics.java:84)
    at org.apache.hadoop.hive.ql.parse.TezCompiler.runStatsAnnotation(TezCompiler.java:466)
    at org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:204)
    at org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:182)
    at org.apache.hadoop.hive.ql.parse.ExecuteStatementAnalyzer.analyzeInternal(ExecuteStatementAnalyzer.java:245)
    at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:332)
    at org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:180)
    at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:332)
    at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:224)
    at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:109)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:508)
    at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:460)
    at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:424)
    at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:418) 

Issue Links

• links to: GitHub Pull Request #5308