Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-6530

Track time spent opening HDFS file handles

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • Impala 2.10.0, Impala 2.11.0
    • Impala 2.12.0
    • Backend
    • None

    Description

      When the HDFS NameNode is overloaded, opening an HDFS file handle can become a time consuming operation and can become the bottleneck for the system. Currently, there is no statistic tracking time spent opening file handles. This limits the ability to determine the extent of the HDFS NameNode problem, especially when given only a single query profile.

      Impala should have a statistic at the HDFS scan node level tracking time spent opening file handles.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            joemcdonnell Joe McDonnell
            joemcdonnell Joe McDonnell
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment