[HIVE-23884] SemanticAnalyzer exception when addressing field with table name in group by

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Unassigned
• Reporter: Rajesh Balamohan
• Created: 20/Jul/20 13:29
• Updated: 09/Sep/20 21:34

Description

explain cbo 
select  `item`.`i_item_id`,
`store`.`s_state`, grouping(s_state) `g_state` from  `tpcds_bin_partitioned_orc_10000`.`store`, `tpcds_bin_partitioned_orc_10000`.`item`
where `store`.`s_state` in ('AL','IN', 'SC', 'NY', 'OH', 'FL')
group by rollup (`item`.`i_item_id`, `s_state`)

CBO PLAN:


HiveProject(i_item_id=[$0], s_state=[$1], g_state=[grouping($2, 0:BIGINT)])
  HiveAggregate(group=[{0, 1}], groups=[[{0, 1}, {0}, {}]], GROUPING__ID=[GROUPING__ID()])
    HiveJoin(condition=[true], joinType=[inner], algorithm=[none], cost=[not available])
      HiveProject(i_item_id=[$1])
        HiveTableScan(table=[[tpcds_bin_partitioned_orc_10000, item]], table:alias=[item])
      HiveProject(s_state=[$24])
        HiveFilter(condition=[IN($24, _UTF-16LE'AL', _UTF-16LE'IN', _UTF-16LE'SC', _UTF-16LE'NY', _UTF-16LE'OH', _UTF-16LE'FL')])
          HiveTableScan(table=[[tpcds_bin_partitioned_orc_10000, store]], table:alias=[store])

 

However, adding fully qualified field name "`store`.`s_state`"" in the second rollup throws SemanticAnalyzer exception

 

explain cbo 
select  `item`.`i_item_id`,
`store`.`s_state`, grouping(s_state) `g_state` from  `tpcds_bin_partitioned_orc_10000`.`store`, `tpcds_bin_partitioned_orc_10000`.`item`
where `store`.`s_state` in ('AL','IN', 'SC', 'NY', 'OH', 'FL')
group by rollup (`item`.`i_item_id`, `store`.`s_state`)

Error: Error while compiling statement: FAILED: RuntimeException [Error 10409]: Expression in GROUPING function not present in GROUP BY (state=42000,code=10409)

Exception: based on 3.x; but mostly should occur in master as well.

Related ticket: https://issues.apache.org/jira/browse/HIVE-15996

Caused by: java.lang.RuntimeException: Expression in GROUPING function not present in GROUP BY
	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$2.post(SemanticAnalyzer.java:3296) ~[hive-exec-3.1xyz]
	at org.antlr.runtime.tree.TreeVisitor.visit(TreeVisitor.java:66) ~[antlr-runtime-3.5.2.jar:3.5.2]
	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.rewriteGroupingFunctionAST(SemanticAnalyzer.java:3305) ~[hive-exec-3.1xyz]
	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:4616) ~[hive-exec-3.1xyz]
	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:4392) ~[hive-exec-3.1xyz]
	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPostGroupByBodyPlan(SemanticAnalyzer.java:11026) ~[hive-exec-3.1xyz]
	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:10965) ~[hive-exec-3.1xyz]
	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11894) ~[hive-exec-3.1xyz]
	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11764) ~[hive-exec-3.1xyz]
	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genOPTree(SemanticAnalyzer.java:12568) ~[hive-exec-3.1xyz]
	at org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:707) ~[hive-exec-3.1xyz]
	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12669) ~[hive-exec-3.1xyz]
	at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:426) ~[hive-exec-3.1xyz]
	at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:288) ~[hive-exec-3.1xyz]
	at org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:170) ~[hive-exec-3.1xyz]
	at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:288) ~[hive-exec-3.1xyz]
	at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:221) ~[hive-exec-3.1xyz]
	at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:104) ~[hive-exec-3.1xyz]
	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:188) ~[hive-exec-3.1xyz]
	at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:598) ~[hive-exec-3.1xyz]
	at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:544) ~[hive-exec-3.1xyz]
	at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:538) ~[hive-exec-3.1xyz]
	at org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:127) ~[hive-exec-3.1xyz

Issue Links

• relates to: HIVE-22253 General task tracking improvements for materialized views