[IMPALA-6680] Consider reserving max 2 I/O buffers per Parquet column when there are multiple columns

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Later
• Affects Version/s: Not Applicable
• Component/s: Backend
• Labels: resource-management
• Target Version: Impala 2.13.0
• Epic Color: ghx-label-3
• Assignee: Tim Armstrong
• Reporter: Tim Armstrong
• Created: 15/Mar/18 22:11
• Updated: 30/Apr/18 16:19
• Resolved: 30/Apr/18 16:19

Description

Reserving 3 * 8MB per column to get triple buffering is minimally useful except in the case when you're scanning a single very large column. 2 * 8MB is enough to overlap compute and I/O, and if you have multiple large columns that will mean that there are multiple I/Os in flight in almost all cases.