Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-23884

SemanticAnalyzer exception when addressing field with table name in group by

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      explain cbo 
      select  `item`.`i_item_id`,
      `store`.`s_state`, grouping(s_state) `g_state` from  `tpcds_bin_partitioned_orc_10000`.`store`, `tpcds_bin_partitioned_orc_10000`.`item`
      where `store`.`s_state` in ('AL','IN', 'SC', 'NY', 'OH', 'FL')
      group by rollup (`item`.`i_item_id`, `s_state`)
      
      CBO PLAN:
      
      
      HiveProject(i_item_id=[$0], s_state=[$1], g_state=[grouping($2, 0:BIGINT)])
        HiveAggregate(group=[{0, 1}], groups=[[{0, 1}, {0}, {}]], GROUPING__ID=[GROUPING__ID()])
          HiveJoin(condition=[true], joinType=[inner], algorithm=[none], cost=[not available])
            HiveProject(i_item_id=[$1])
              HiveTableScan(table=[[tpcds_bin_partitioned_orc_10000, item]], table:alias=[item])
            HiveProject(s_state=[$24])
              HiveFilter(condition=[IN($24, _UTF-16LE'AL', _UTF-16LE'IN', _UTF-16LE'SC', _UTF-16LE'NY', _UTF-16LE'OH', _UTF-16LE'FL')])
                HiveTableScan(table=[[tpcds_bin_partitioned_orc_10000, store]], table:alias=[store])
      

       

      However, adding fully qualified field name "`store`.`s_state`"" in the second rollup throws SemanticAnalyzer exception

       

      explain cbo 
      select  `item`.`i_item_id`,
      `store`.`s_state`, grouping(s_state) `g_state` from  `tpcds_bin_partitioned_orc_10000`.`store`, `tpcds_bin_partitioned_orc_10000`.`item`
      where `store`.`s_state` in ('AL','IN', 'SC', 'NY', 'OH', 'FL')
      group by rollup (`item`.`i_item_id`, `store`.`s_state`)
      
      Error: Error while compiling statement: FAILED: RuntimeException [Error 10409]: Expression in GROUPING function not present in GROUP BY (state=42000,code=10409)
      
      

      Exception: based on 3.x; but mostly should occur in master as well.

      Related ticket: https://issues.apache.org/jira/browse/HIVE-15996

      Caused by: java.lang.RuntimeException: Expression in GROUPING function not present in GROUP BY
      	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$2.post(SemanticAnalyzer.java:3296) ~[hive-exec-3.1xyz]
      	at org.antlr.runtime.tree.TreeVisitor.visit(TreeVisitor.java:66) ~[antlr-runtime-3.5.2.jar:3.5.2]
      	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.rewriteGroupingFunctionAST(SemanticAnalyzer.java:3305) ~[hive-exec-3.1xyz]
      	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:4616) ~[hive-exec-3.1xyz]
      	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:4392) ~[hive-exec-3.1xyz]
      	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPostGroupByBodyPlan(SemanticAnalyzer.java:11026) ~[hive-exec-3.1xyz]
      	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:10965) ~[hive-exec-3.1xyz]
      	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11894) ~[hive-exec-3.1xyz]
      	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11764) ~[hive-exec-3.1xyz]
      	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genOPTree(SemanticAnalyzer.java:12568) ~[hive-exec-3.1xyz]
      	at org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:707) ~[hive-exec-3.1xyz]
      	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12669) ~[hive-exec-3.1xyz]
      	at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:426) ~[hive-exec-3.1xyz]
      	at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:288) ~[hive-exec-3.1xyz]
      	at org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:170) ~[hive-exec-3.1xyz]
      	at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:288) ~[hive-exec-3.1xyz]
      	at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:221) ~[hive-exec-3.1xyz]
      	at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:104) ~[hive-exec-3.1xyz]
      	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:188) ~[hive-exec-3.1xyz]
      	at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:598) ~[hive-exec-3.1xyz]
      	at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:544) ~[hive-exec-3.1xyz]
      	at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:538) ~[hive-exec-3.1xyz]
      	at org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:127) ~[hive-exec-3.1xyz
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                rajesh.balamohan Rajesh Balamohan
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated: