Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-9832

Merge join followed by union and a map join in hive on tez fails.

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 1.0.0, 1.2.0, 1.1.0
    • Fix Version/s: 1.2.0
    • Component/s: Tez
    • Labels:
      None

      Description

      select a.key, b.value from (select x.key as key, y.value as value from
      srcpart x join srcpart y on (x.key = y.key)
      union all
      select key, value from srcpart z) a join src b on (a.value = b.value);
      
      TaskAttempt 3 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: Hive Runtime Error while closing operators: null
              at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:186)
              at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:138)
              at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:324)
              at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
              at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
              at java.security.AccessController.doPrivileged(Native Method)
              at javax.security.auth.Subject.doAs(Subject.java:415)
              at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
              at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
              at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
              at java.util.concurrent.FutureTask.run(FutureTask.java:262)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
              at java.lang.Thread.run(Thread.java:745)
      Caused by: java.lang.RuntimeException: Hive Runtime Error while closing operators: null
              at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.close(ReduceRecordProcessor.java:214)
              at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:177)
              ... 13 more
      Caused by: java.lang.NullPointerException
              at org.apache.hadoop.hive.ql.exec.MapJoinOperator.closeOp(MapJoinOperator.java:317)
              at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:598)
              at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:610)
              at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:610)
              at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:610)
              at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:610)
              at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.close(ReduceRecordProcessor.java:196)
              ... 14 more
      ]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1425055721029_0048_4_09 [Reducer 5] killed/failed due to:null]
      Vertex killed, vertexName=Reducer 7, vertexId=vertex_1425055721029_0048_4_11, diagnostics=[Vertex received Kill while in RUNNING state., Vertex killed as other vertex failed. failedTasks:0, Vertex vertex_1425055721029_0048_4_11 [Reducer 7] killed/failed due to:null]
      Vertex killed, vertexName=Reducer 4, vertexId=vertex_1425055721029_0048_4_07, diagnostics=[Vertex received Kill while in RUNNING state., Vertex killed as other vertex failed. failedTasks:0, Vertex vertex_1425055721029_0048_4_07 [Reducer 4] killed/failed due to:null]
      DAG failed due to vertex failure. failedVertices:1 killedVertices:2
      FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask
      

        Attachments

        1. HIVE-9832.1.patch
          589 kB
          Vikram Dixit K
        2. HIVE-9832.2.patch
          13 kB
          Vikram Dixit K
        3. HIVE-9832.3.patch
          14 kB
          Vikram Dixit K
        4. HIVE-9832.4.patch
          19 kB
          Vikram Dixit K

          Issue Links

            Activity

              People

              • Assignee:
                vikram.dixit Vikram Dixit K
                Reporter:
                vikram.dixit Vikram Dixit K
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: