Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-9103 Tracking spark's memory usage
  3. SPARK-9107

Include memory usage for each job & stage

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Won't Fix
    • None
    • None
    • Spark Core, Web UI
    • None

    Description

      In spark-9104, the memory usage is showed as running time memory usage, which means, we can only see the current status of the memory consumption, just in the same way how the "Storage" Tab works, it would not store the previous status, and the same situation for history server WebUI.

      The target for this issue is to show in different finished stages or in different finished jobs, what is the memory usage status when/before the job/stage completes. Also we can give out the maximum and minimum memory size used during each job/stage.

      Attachments

        Activity

          People

            Unassigned Unassigned
            liyezhang556520 Zhang, Liye
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: