[PARQUET-1765] Invalid filteredRowCount in InternalParquetRecordReader

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.11.0
• Fix Version/s: 1.12.0, 1.11.1
• Component/s: parquet-mr
• Labels: pull-request-available
• Assignee: Gabor Szadovszky
• Reporter: Gabor Szadovszky
• Created: 13/Jan/20 16:24
• Updated: 23/Jun/24 03:31
• Resolved: 16/Jan/20 13:31

Description

The record count is retrieved before setting the projection schema so the value might be invalid if the projection impacts the filter.

In normal cases it does not cause any issue because the record filter will filter correctly only that we are filtering the records one-by-one instead of dropping the related pages.

Issue Links

• blocks: PARQUET-1774 Release parquet 1.11.1
• links to: GitHub Pull Request #747