[IMPALA-10147] Avoid getting a file handle for data cache hits

• Type: Improvement
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 4.0.0
• Fix Version/s: Impala 4.0.0
• Component/s: Backend
• Target Version: Impala 4.0.0
• Epic Color: ghx-label-2
• Assignee: Riza Suminto
• Reporter: Joe McDonnell
• Created: 04/Sep/20 23:43
• Updated: 06/Nov/22 00:23
• Resolved: 18/Feb/21 03:01

Description

When reading from the data cache, the DiskIo thread first gets a file handle, then it checks the data cache for a hit. If there is a cache hit, then the file handle is not actually used. It is only used if there is a cache miss. There is no real reason to have the file handle open for cache hits. It doesn't really serve any additional purpose, and it adds overhead to cache hits.

For platforms that do not have the file handle cache, this can be a significant overhead.

We should only open the file handle after we have checked the data cache and know that we need to read from regular storage.

Issue Links

• is related to: IMPALA-11704 Remote Ozone scans are slow even after data cache warmup