[IMPALA-5347] Parquet scanner has a lot of small CPU inefficiencies

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: Impala 2.9.0
• Fix Version/s: Impala 2.9.0
• Component/s: Backend
• Labels: parquet performance
• Target Version: Product Backlog
• Epic Color: ghx-label-4
• Assignee: Tim Armstrong
• Reporter: Tim Armstrong
• Created: 22/May/17 15:00
• Updated: 07/Feb/23 11:13
• Resolved: 25/May/17 15:48

Description

I spent some time looking at the parquet scanner in perf top. There are a lot of cases where the code is inefficient in ways that are easily fixed. Together this could add up to a significant perf win for scans.

The assembly of the core MaterializeValueBatch() loop has a lot of obvious inefficiency:

• Many loads from memory of values that are constant within the loop
• The generated bit unpacking and dictionary decoding code has a lot of inefficiency, e.g. a complicated bounds check
• Hot functions like DictDecoder::Get() are not inlined.

A lot of time is also spent on some scans calling memset() on one or two bytes inside InitTuple().