Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-27019

Spark UI's SQL tab shows inconsistent values

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.4.0
    • 2.4.1, 3.0.0
    • SQL, Web UI
    • None

    Description

      Since 2.4.0, I am frequently seeing broken outputs in the SQL tab of the Spark UI, where submitted/duration make no sense, description has the ID instead of the actual description.

      Clicking on the link to open a query, the SQL plan is missing as well.

      I have tried to increase `spark.scheduler.listenerbus.eventqueue.capacity` to very large values like 30k out of paranoia that we may have too many events, but to no avail. I have not identified anything particular that leads to that: it doesn't occur in all my jobs, but it does occur in a lot of them still.

      Attachments

        1. screenshot-spark-ui-list.png
          229 kB
          peay
        2. screenshot-spark-ui-details.png
          26 kB
          peay
        3. Screenshot from 2019-03-01 21-31-48.png
          155 kB
          shahid
        4. query-job-1.png
          60 kB
          peay
        5. query-1-list.png
          29 kB
          peay
        6. query-1-details.png
          26 kB
          peay
        7. application_1550040445209_4748
          80 kB
          peay

        Issue Links

          Activity

            People

              shahid shahid
              peay peay
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: