Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-6232

Short circuit reads disabled when using Impala HDFS file handle cache

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: Impala 2.10.0
    • Fix Version/s: Impala 2.11.0
    • Component/s: Backend
    • Labels:
      None

      Description

      In Impala 2.10, the HDFS file handle cache was enabled by default. However, testing has revealed that in cases where files are overwritten or appended, the file handle can encounter an error that causes HDFS to disable short circuit reads for 10 minutes. See HDFS-12528.

      Due to this performance impact and the associated unpredictability, Impala should disable the file handle cache by default until this issue is resolved.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                joemcdonnell Joe McDonnell
                Reporter:
                joemcdonnell Joe McDonnell
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: