Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-24501

Add metrics for Mesos Driver and Dispatcher

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: In Progress
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 2.3.1
    • Fix Version/s: None
    • Component/s: Mesos
    • Labels:
      None
    • Flags:
      Patch

      Description

      The Mesos Dispatcher currently only has three gauge metrics, for the number of waiting, launched, and retrying drivers, defined in MesosClusterSchedulerSource.scala. Meanwhile, the Mesos Driver doesn't have any metrics defined at all.

      Implement additional metrics for both the Mesos Dispatcher and Mesos Driver, including e.g. types of events received, how long it takes jobs to run, for use by operators to monitor and/or diagnose their Spark environment.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              nickbp Nicholas Parker
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: