[HIVE-14232] Kryo: Perflogger is not transient in MapJoinOperator

• Type: Bug
• Status: Open
• Priority: Blocker
• Resolution: Unresolved
• Affects Version/s: 2.2.0
• Component/s: Hive
• Assignee: Nita Dembla Assign to me
• Reporter: Gopal Vijayaraghavan
• Created: 13/Jul/16 23:21
• Updated: 20/Jul/16 00:45

Description

  private final PerfLogger perfLogger = SessionState.getPerfLogger();

This produces a new copy of HiveConf into the plan in serialized form.

classes (sun.misc.Launcher$AppClassLoader)
classloader (java.security.ProtectionDomain)
context (java.security.AccessControlContext)
acc (org.apache.hadoop.hive.ql.exec.UDFClassLoader)
classLoader (org.apache.hadoop.hive.conf.HiveConf)
conf (org.apache.hadoop.hive.common.metrics.metrics2.CodahaleMetrics)
metrics (org.apache.hadoop.hive.common.metrics.metrics2.CodahaleMetrics$CodahaleMetricsScope)
openScopes (org.apache.hadoop.hive.ql.log.PerfLogger)
perfLogger (org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinInnerLongOperator)
childOperators (org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinInnerLongOperator)
childOperators (org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator)
childOperators (org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator)
childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
        at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:101)
        at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:518)
        at org.apache.hive.com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:552)
        at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:80)
        at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:518)
        at org.apache.hive.com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:628)
        at org.apache.hive.com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.write(DefaultArraySerializers.java:366)
        at org.apache.hive.com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.write(DefaultArraySerializers.java:307)
        at org.apache.hive.com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:552)
        at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:80)
        at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:518)
        at org.apache.hive.com.esotericsoftware.kryo.Kryo.writeObjectOrNull(Kryo.java:606)
        at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:87)
        at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:518)
        at org.apache.hive.com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:552)
        at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:80)
        at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:518)
        at org.apache.hive.com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:552)
        at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:80)
        at org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:518)
        at org.apache.hive.com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:552)
        at org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:80)

Issue Links

• duplicates: HIVE-14293 PerfLogger.openScopes should be transient