Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-31967

Loading jobs UI page takes 40 seconds

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • 2.4.6, 3.0.1
    • 2.4.7, 3.0.1, 3.1.0
    • Web UI
    • None

    Description

      In the latest master branch, I find that the job list page becomes very slow.
      To reproduce in local setup:

      spark.read.parquet("/tmp/p1").createOrReplaceTempView("t1")
      spark.read.parquet("/tmp/p2").createOrReplaceTempView("t2")
      (1 to 1000).map(_ =>  spark.sql("select * from t1, t2 where t1.value=t2.value").show())
      

      And that, open live UI: http://localhost:4040/
      The loading time is about 40 seconds.

      If we comment out the function call for `drawApplicationTimeline`, then the loading time is around 1 second.

      Attachments

        1. load_time.jpeg
          15 kB
          Gengliang Wang
        2. profile.png
          65 kB
          Gengliang Wang

        Activity

          People

            Gengliang.Wang Gengliang Wang
            Gengliang.Wang Gengliang Wang
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: