Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-9103 Tracking spark's memory usage
  3. SPARK-21934

Expose Netty memory usage via Metrics System

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.3.0
    • Fix Version/s: 2.3.0
    • Component/s: Spark Core
    • Labels:
      None

      Description

      This is a follow-up work of SPARK-9104 to expose the Netty memory usage to MetricsSystem. My initial thought is to only expose Shuffle memory usage, since shuffle is a major part of memory usage in network communication compared to RPC, file server, block transfer.

      If user wants to also expose Netty memory usage for other modules, we could add more metrics later.

        Attachments

          Activity

            People

            • Assignee:
              jerryshao Saisai Shao
              Reporter:
              jerryshao Saisai Shao
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: