Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-32135

Show Spark Driver name on Spark history web page

    XMLWordPrintableJSON

    Details

    • Type: Wish
    • Status: In Progress
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 2.4.5
    • Fix Version/s: None
    • Component/s: Spark Core
    • Labels:
      None

      Description

      Our service dynamically creates short-lived YARN clusters in cloud. Spark applications run on these dynamically created clusters. Log data for these applications is stored on a remote file-system. We configure a static instance of SparkHistoryServer to view information on jobs that ran on these clusters.

      Since we are using a single History Server for multiple clusters, it will be useful for users to have information on where the jobs were executed. We would like to display this information on the main web-page, instead of having users go through multiple links to retrieve this information.

        Attachments

        1. image-2020-09-02-12-37-55-860.png
          126 kB
          Gaurangi Saxena

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              gaurangi Gaurangi Saxena
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated: