[IMPALA-11685] Slot memory sharing between struct and field not working if the field is also a struct

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Component/s: Frontend
• Epic Color: ghx-label-3
• Assignee: Daniel Becker
• Reporter: Daniel Becker
• Created: 25/Oct/22 15:26
• Updated: 31/Oct/22 04:48
• Resolved: 27/Oct/22 16:45

Description

IMPALA-10838 introduced that if a struct and one of its fields are both present in the select list, no extra slot is generated in the row for the struct field but the memory of the struct is reused, i.e. the row size is the same as when only the struct is queried. It works when the struct field is a primitive type:

explain select id, outer_struct from functional_orc_def.complextypes_nested_structs;
row-size=64B

explain select id, outer_struct, outer_struct.str from functional_orc_def.complextypes_nested_structs;
row-size=64B

However, it does not if the child is itself a struct:

explain select id, outer_struct, outer_struct.inner_struct3 from functional_orc_def.complextypes_nested_structs;
row-size=80B

This is because struct slot descriptors are registered before others so that it is easier to reuse the slot memory of the struct fields, but struct slot descriptors among themselves are sorted in the wrong order (see https://github.com/apache/impala/blob/c12ac6c27b2df1eae693b44c157d65499f491d21/fe/src/main/java/org/apache/impala/analysis/SelectStmt.java#L340).

Issue Links

• is related to: IMPALA-10838 Error when struct returned from WITH() and used in an ORDER BY
• relates to: IMPALA-11692 Struct slot memory sharing involving select * not working properly