Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
0.14.0
-
None
Description
The following describes how to produce the bug:
hive> desc test2; name string age int hive> select * from test2; 6666666666666666666 NULL 5555555555555555555 NULL tom 15 john NULL mayr 40 30 NULL hive> create table test3(name string) partitioned by (age int); hive> from test2 insert overwrite table test3 partition(age) select test2.name, test2.age; Loading data to table default.test3 partition (age=null) Loading partition {age=40} Loading partition {age=__HIVE_DEFAULT_PARTITION__} Loading partition {age=30} Loading partition {age=15} Partition default.test3{age=15} stats: [numFiles=1, numRows=1, totalSize=4, rawDataSize=3] Partition default.test3{age=30} stats: [numFiles=1, numRows=1, totalSize=1, rawDataSize=0] Partition default.test3{age=40} stats: [numFiles=1, numRows=1, totalSize=5, rawDataSize=4] Partition default.test3{age=__HIVE_DEFAULT_PARTITION__} stats: [numFiles=1, numRows=4, totalSize=46, rawDataSize=42] hive> analyze table test3 partition(age) compute statistics; ... Task with the most failures(4): ----- Diagnostic Messages for this Task: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"name":"6666666666666666666","age":null,"raw__data__size":19} at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:195) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:417) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1499) at org.apache.hadoop.mapred.Child.main(Child.java:262) Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"name":"6666666666666666666","age":null,"raw__data__size":19} at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:549) at org.apache.hado FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
The following is the stack trace in mapper log:
2014-04-28 15:39:25,073 FATAL org.apache.hadoop.hive.ql.exec.mr.ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"name":"6666666666666666666","age":null,"raw__data__size":19} at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:549) at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:177) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:417) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1499) at org.apache.hadoop.mapred.Child.main(Child.java:262) Caused by: java.lang.NullPointerException at org.apache.hadoop.hive.ql.exec.TableScanOperator.gatherStats(TableScanOperator.java:149) at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:90) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:796) at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:539) ... 9 more