Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-9141

HiveOnTez: mix of union all, distinct, group by generates error

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.15.0
    • Fix Version/s: 1.0.0
    • Component/s: Tez
    • Labels:
      None

      Description

      Here is the way to produce it:
      in Hive q test setting (with src table)
      set hive.execution.engine=tez;

      SELECT key, value FROM
      (
      SELECT key, value FROM src

      UNION ALL

      SELECT key, key as value FROM

      (
      SELECT distinct key FROM (

      SELECT key, value FROM
      (SELECT key, value FROM src
      UNION ALL
      SELECT key, value FROM src
      )t1
      group by key, value
      )t2
      )t3

      )t4
      group by key, value;

      will generate

      2014-12-16 23:19:13,593 ERROR ql.Driver (SessionState.java:printError(834)) - FAILED: ClassCastException org.apache.hadoop.hive.ql.plan.MapWork cannot be cast to org.apache.hadoop.hive.ql.plan.ReduceWork
      java.lang.ClassCastException: org.apache.hadoop.hive.ql.plan.MapWork cannot be cast to org.apache.hadoop.hive.ql.plan.ReduceWork
      at org.apache.hadoop.hive.ql.parse.GenTezWork.process(GenTezWork.java:361)
      at org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
      at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:94)
      at org.apache.hadoop.hive.ql.parse.GenTezWorkWalker.walk(GenTezWorkWalker.java:87)
      at org.apache.hadoop.hive.ql.parse.GenTezWorkWalker.walk(GenTezWorkWalker.java:103)
      at org.apache.hadoop.hive.ql.parse.GenTezWorkWalker.walk(GenTezWorkWalker.java:103)
      at org.apache.hadoop.hive.ql.parse.GenTezWorkWalker.walk(GenTezWorkWalker.java:103)
      at org.apache.hadoop.hive.ql.parse.GenTezWorkWalker.walk(GenTezWorkWalker.java:103)
      at org.apache.hadoop.hive.ql.parse.GenTezWorkWalker.startWalking(GenTezWorkWalker.java:69)
      at org.apache.hadoop.hive.ql.parse.TezCompiler.generateTaskTree(TezCompiler.java:368)
      at org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:202)
      at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10202)
      at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:224)
      at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:419)
      at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:305)
      at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1107)
      at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1155)
      at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1044)
      at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1034)
      at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:206)
      at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:158)
      at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:369)
      at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:304)
      at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:834)
      at org.apache.hadoop.hive.cli.TestMiniTezCliDriver.runTest(TestMiniTezCliDriver.java:136)
      at org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_uniontez2(TestMiniTezCliDriver.java:120)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                navis Navis
                Reporter:
                pxiong Pengcheng Xiong
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: