[IMPALA-6232] Short circuit reads disabled when using Impala HDFS file handle cache

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: Impala 2.10.0
• Fix Version/s: Impala 2.11.0
• Component/s: Backend
• Target Version: Impala 2.11.0
• Epic Color: ghx-label-3
• Assignee: Joe McDonnell
• Reporter: Joe McDonnell
• Created: 22/Nov/17 00:46
• Updated: 05/Dec/17 21:38
• Resolved: 05/Dec/17 21:38

Description

In Impala 2.10, the HDFS file handle cache was enabled by default. However, testing has revealed that in cases where files are overwritten or appended, the file handle can encounter an error that causes HDFS to disable short circuit reads for 10 minutes. See HDFS-12528.

Due to this performance impact and the associated unpredictability, Impala should disable the file handle cache by default until this issue is resolved.

Issue Links

• is related to: HDFS-12528 Add an option to not disable short-circuit reads on failures