Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-23206 Additional Memory Tuning Metrics
  3. SPARK-26341

Expose executor memory metrics at the stage level, in the Stages tab

Attach filesAttach ScreenshotVotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.1
    • Fix Version/s: 3.1.0, 3.2.0
    • Component/s: Spark Core, Web UI
    • Labels:
      None

      Description

      Sub-task SPARK-23431 will add stage level executor memory metrics (peak values for each stage, and peak values for each executor for the stage). This information should also be exposed the the web UI, so that users can see which stages are memory intensive.

        Attachments

          Activity

            People

            • Assignee:
              angerszhuuu angerszhu
              Reporter:
              elu Edward Lu

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment