[IMPALA-10375] Lock down which filesystem types use the file handle cache

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 4.0.0
• Fix Version/s: Impala 4.2.0
• Component/s: Backend
• Target Version: Impala 4.2.0
• Epic Color: ghx-label-5
• Assignee: Joe McDonnell
• Reporter: Joe McDonnell
• Created: 03/Dec/20 20:59
• Updated: 31/Aug/22 19:46
• Resolved: 31/Aug/22 19:44

Description

When determining whether to cache file handles, the code uses this check:

bool use_file_handle_cache = false;
if (is_file_handle_caching_enabled() &&
    (expected_local_ ||
     (FLAGS_cache_remote_file_handles && disk_id_ == io_mgr_->RemoteDfsDiskId()) ||
     (FLAGS_cache_s3_file_handles && disk_id_ == io_mgr_->RemoteS3DiskId()) ||
     (FLAGS_cache_abfs_file_handles && disk_id_ == io_mgr_->RemoteAbfsDiskId())) {
  use_file_handle_cache = true;
}

If expected_local_ is specified, then we always would allow caching, regardless of filesystem type. While expected_local_ should be false for remote filesystems, the code should be more defensive and only use the file handle cache for specific supported filesystems. In particular, we currently don't support caching Ozone file handles, but if this somehow considered Ozone local, then it would cache them.

Issue Links

• is related to: IMPALA-10214 Ozone support for file handle cache
• is related to: IMPALA-11457 Ozone parallelism reduced when backends are co-located
• is related to: IMPALA-11448 Always assign Ozone I/O to remote thread group to improve performance
• links to: Gerrit Code Review