Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-17243

Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.0.0
    • 2.0.1, 2.1.0
    • Web UI
    • None
    • Linux

    Description

      The summary page of Spark 2.0 history server web UI keep displaying "Loading history summary..." all the time and crashes the browser when there are more than 10K application history event logs on HDFS.

      I did some investigation, "historypage.js" file sends a REST request to /api/v1/applications endpoint of history server REST endpoint and gets back json response. When there are more than 10K applications inside the event log directory it takes forever to parse them and render the page. When there are only hundreds or thousands of application history it is running fine.

      Attachments

        Activity

          People

            ajbozarth Alex Bozarth
            wgtmac Gang Wu
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: