[IMPALA-11147] Min/max filtering crashes on Parquet file that contains partition columns

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 4.1.0
• Component/s: Backend
• Epic Color: ghx-label-3
• Assignee: Zoltán Borók-Nagy
• Reporter: Zoltán Borók-Nagy
• Created: 22/Feb/22 15:17
• Updated: 24/Feb/22 10:34
• Resolved: 24/Feb/22 10:34

Description

Impala can crash on a Parquet file that contains the partition columns.

Data files usually don't contain the partition columns, so Impala don't expect to find such columns in the data files. Unfortunately min/max filtering generates a SEGFAULT when the partition column is present in the data files.

It happens because FindSkipRangesForPagesWithMinMaxFilters() tries to retrieve the Parquet schema element for a given slot descriptor. When the slot descriptor refers to a partition column, we usually don't find a schema element so we don't try to skip pages.

But when the partition column is present in the data file, the code tries to calculate the filtered pages in the column. It uses the column reader object corresponding to the column, but this is null for partition columns, hence we get a SEGFAULT.

The code shouldn't do anything at the page-level for partition columns, as the data in such columns are the same for the whole file and it is already filtered at a higher level.