Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-18369

NullPointerException when void convert to varchar using Tez

    Details

    • Flags:
      Important

      Description

      NullPointerException when void convert to varchar using Tez
      Just like follow testSQL. we can run it using MR ,but we get a NullPointerException when using Tez. we found it failed in SQL compiling when doing constant propagation optimizer.

      testSQL:
      create table testTezVarchar(a varchar(10),b varchar(10));
      create table testTezVoid as select null as a1,null as b1 from events5;
      set hive.execution.engine=mr;
      insert into testTezVarchar select a1 as a,b1 as b from testTezVoid;
      set hive.execution.engine=tez;
      insert into testTezVarchar select a1 as a,b1 as b from testTezVoid;

      Exception:
      Caused by: java.lang.NullPointerException
      at org.apache.hadoop.hive.common.type.HiveBaseChar.enforceMaxLength(HiveBaseChar.java:44)
      at org.apache.hadoop.hive.serde2.io.HiveVarcharWritable.set(HiveVarcharWritable.java:65)
      at org.apache.hadoop.hive.serde2.io.HiveVarcharWritable.set(HiveVarcharWritable.java:61)
      at org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableHiveVarcharObjectInspector.set(WritableHiveVarcharObjectInspector.java:130)
      at org.apache.hadoop.hive.serde2.objectinspector.primitive.PrimitiveObjectInspectorConverter$HiveVarcharConverter.convert(PrimitiveObjectInspectorConverter.java:490)
      at org.apache.hadoop.hive.ql.udf.generic.GenericUDFToVarchar.evaluate(GenericUDFToVarchar.java:84)
      at org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.evaluateFunction(ConstantPropagateProcFactory.java:516)
      at org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.foldExpr(ConstantPropagateProcFactory.java:228)
      at org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory.access$000(ConstantPropagateProcFactory.java:91)
      at org.apache.hadoop.hive.ql.optimizer.ConstantPropagateProcFactory$ConstantPropagateSelectProc.process(ConstantPropagateProcFactory.java:718)
      at org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
      at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:94)
      at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:78)
      at org.apache.hadoop.hive.ql.optimizer.ConstantPropagate$ConstantPropagateWalker.walk(ConstantPropagate.java:142)
      at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:109)
      at org.apache.hadoop.hive.ql.optimizer.ConstantPropagate.transform(ConstantPropagate.java:112)
      at org.apache.hadoop.hive.ql.parse.TezCompiler.runDynamicPartitionPruning(TezCompiler.java:311)
      at org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:117)
      at org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:101)

      Now we just set "hive.optimize.constant.propagation=false" to avoid this issue,does anybody knows how to solve it ?

        Attachments

        1. testSql.sql
          0.3 kB
          ZhongyuWang
        2. hive.log
          5 kB
          ZhongyuWang

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                ZhongyuWang ZhongyuWang
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Due:
                  Created:
                  Updated:
                  Resolved: