Details
-
Type:
Bug
-
Status: Resolved
-
Priority:
Critical
-
Resolution: Fixed
-
Affects Version/s: Impala 2.9.0
-
Fix Version/s: Impala 2.9.0
-
Component/s: Backend
-
Labels:None
-
Target Version:
-
Epic Color:ghx-label-8
Description
The Disk IO Manager has customized support for S3 and remote HDFS that allows for these to use a separate queue and have a customized number of IO threads. Currently, ADLS does not have this support.
Based on the code in DiskIoMgr::Init and DiskIoMgr::AssignQueue, IOs for ADLS would currently be put in local disk queues. Since local disks are considered rotational unless we can confirm otherwise by looking at the /sys filesystem, this means that THREADS_PER_ROTATIONAL_DISK=1 will apply as the thread count.
This reduced thread count may hurt ADLS performance.