Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-9479

Include GC time in runtime profiles

    XMLWordPrintableJSON

Details

    • Task
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • None
    • None
    • None
    • None
    • ghx-label-9

    Description

      The JvmPauseMonitor prints out logs whenever it detects an excessive amount of time being spent in GC. However, these log lines can often go unnoticed, it would be useful to include some GC related information in the runtime profiles.

      This is useful for diagnosing:

      • Issues with Java UDFs that spend a lot of time in GC
      • GC issues on the Coordinator from the fe/ code
      • Some S3 operations could potentially be GC intensive - e.g. S3A block output stream

      I'm not sure there is a way to track GC per query, since GC happens globally inside the JVM. There are a few ways to get GC information into the profile:

      • If the JvmPauseMonitor detects a GC pause it can insert a warning in the profiles of all running queries
      • JMX metrics can be used to detect how much time was spent in GC from when a fragment began to when it ended

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              stakiar Sahil Takiar
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: