Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-27019

Spark UI's SQL tab shows inconsistent values

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.4.0
    • Fix Version/s: 2.4.1, 3.0.0
    • Component/s: SQL, Web UI
    • Labels:
      None

      Description

      Since 2.4.0, I am frequently seeing broken outputs in the SQL tab of the Spark UI, where submitted/duration make no sense, description has the ID instead of the actual description.

      Clicking on the link to open a query, the SQL plan is missing as well.

      I have tried to increase `spark.scheduler.listenerbus.eventqueue.capacity` to very large values like 30k out of paranoia that we may have too many events, but to no avail. I have not identified anything particular that leads to that: it doesn't occur in all my jobs, but it does occur in a lot of them still.

        Attachments

        1. application_1550040445209_4748
          80 kB
          peay
        2. query-1-details.png
          26 kB
          peay
        3. query-1-list.png
          29 kB
          peay
        4. query-job-1.png
          60 kB
          peay
        5. Screenshot from 2019-03-01 21-31-48.png
          155 kB
          shahid
        6. screenshot-spark-ui-details.png
          26 kB
          peay
        7. screenshot-spark-ui-list.png
          229 kB
          peay

          Issue Links

            Activity

              People

              • Assignee:
                shahid shahid
                Reporter:
                peay peay
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: