[IMPALA-5169] Parallelise read I/O of BufferPool::Pin()

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 2.9.0
• Fix Version/s: Impala 2.9.0
• Component/s: Backend
• Target Version: Impala 2.9.0
• Epic Color: ghx-label-6
• Assignee: Tim Armstrong
• Reporter: Tim Armstrong
• Created: 05/Apr/17 18:33
• Updated: 11/May/17 19:58
• Resolved: 11/May/17 19:58

Description

Currently read I/O in BufferPool is synchronous. In some cases this can lead to poor resource utilisation and I/O throughput, because:

• We don't dispatch parallel reads to multiple scratch disks or high-throughput SSDs
• Issuing reads of contiguous scratch ranges at the same time improves the odds that the second read can be served without a disk seek or by the disks internal cache.

• Expose a batched Pin() interface that can pin multiple buffers at the same time
• Expose an asynchronous Pin() interface that can start the read, and allow the client to wait for it.

The first alternative is probably simplest.

Issue Links

• relates to: IMPALA-3200 Replace BufferedBlockMgr with new buffer pool