Details
-
Improvement
-
Status: Open
-
Major
-
Resolution: Unresolved
-
Impala 4.1.0
-
None
-
ghx-label-13
Description
IMPALA-9496 introduced querying structs in the select list even for Parquet tables. However, it has a restriction that in this case late materialization (LM) can't be used.
The reason is that LM expects the column readers to be used through the batched reading interface, however, currently struct column readers use the non-batched reading interface of its children. As a result after reading, the column readers are not in a state as SkipRows() of LM expects and then results in a query failure because it's not able to skip the rows for non-filter readers.
Once IMPALA-11363 is implemented and the struct will also use the ReadValueBatch() interface of its children then late materialization could be turned on even if structs are in the select list.
Attachments
Issue Links
- depends upon
-
IMPALA-9496 Allow Struct type in SELECT list for Parquet tables
- Resolved
-
IMPALA-11363 Use ReadValueBatch() when the members of Parquet StructColumnReader
- Open