Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
1.2.1
-
None
-
None
Description
I have seen some instances where removeField in SerializationUtilities is not working
removeField(kryo, Operator.class, "colExprMap"); removeField(kryo, AbstractOperatorDesc.class, "statistics");
These 2 fields are not expected to be serialized but some stacktraces where custom UDFs are involved these fields are observed in the stacktrace.
genericUDF (org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc) chidren (org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc) colExprMap (org.apache.hadoop.hive.ql.exec.SelectOperator) childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator) aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork) at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125) ...(snip)... at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:672) at org.apache.hadoop.hive.ql.exec.Utilities.deserializeObjectByKryo(Utilities.java:1173) at org.apache.hadoop.hive.ql.exec.Utilities.deserializePlan(Utilities.java:1062) at org.apache.hadoop.hive.ql.exec.Utilities.deserializePlan(Utilities.java:1076)
This needs further investigation.
Attachments
Issue Links
- is related to
-
HIVE-15218 Kyro Exception on subsequent run of a query in LLAP mode
- Resolved