[IMPALA-5704] Parallelise read I/O for spilling operators where it is currently sequential

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: Impala 2.10.0
• Component/s: Backend
• Labels: performance
• Target Version: Product Backlog
• Epic Color: ghx-label-7
• Assignee: Unassigned
• Reporter: Tim Armstrong
• Created: 24/Jul/17 17:57
• Updated: 14/May/20 18:20

Description

After IMPALA-3200 goes in, we can make improvements to the performance of spilling operators by exploiting the new async pin behaviour of IMPALA-5169. There are several known cases where read I/O is not effectively overlapped with compute, and multiple disks are not used effectively by doing multiple I/Os concurrently

Sub-Tasks

• Parallelise read I/O by prefetching pages when iterating over unpinned BufferedTupleStream: Open
• Parallelise read I/O in sorter: Resolved