Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-7157

Avoid unnecessarily pretty printing profiles per fragment instance

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • Impala 3.1.0
    • Distributed Exec
    • ghx-label-9

    Description

      In SendReport(), if VLOG_FILE_IS_ON is 'true' (which is not the most verbose logging level, but is higher than default), we pretty print the profile for every fragment instance, which is a very expensive operation, as serializing the profile is non-trivial (look at RuntimeProfile::PrettyPrint()), and printing large amounts of information to the logs isn't cheap as well. Lastly, it is very noisy.

      This seems unnecessary since this will not benefit us, as all the profiles are merged at the coordinator side. We could argue that this might be necessary when an executor fails to send the profile to the coordinator, but that signifies a network issue which will not be reflected in the profile of any fragment instance.

      This will help reduce noise in the logs when the log level is bumped up to find other real issues that VLOG_FILE can help with.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            sailesh Sailesh Mukil
            sailesh Sailesh Mukil
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment