[HIVE-21400] Vectorization: LazyBinarySerializeWrite allocates Field() within the loop

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 4.0.0-alpha-1
• Component/s: Vectorization
• Assignee: Gopal Vijayaraghavan
• Reporter: Gopal Vijayaraghavan
• Created: 07/Mar/19 05:57
• Updated: 17/Nov/22 08:50
• Resolved: 24/Apr/19 22:37

Description

GC thrash from an unexpected source in ReduceSinkOperator.

org.apache.hadoop.hive.serde2.lazybinary.fast.LazyBinarySerializeWrite.resetWithoutOutput(LazyBinarySerializeWrite.java:136)
        at org.apache.hadoop.hive.serde2.lazybinary.fast.LazyBinarySerializeWrite.reset(LazyBinarySerializeWrite.java:132)
        at org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkUniformHashOperator.process(VectorReduceSinkUniformHashOperator.java:180)

GC space is getting thrashed by the

    root = new Field(STRUCT);

for every row.