Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-2035

Make a stage's call stack available on the UI

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • 1.1.0
    • Web UI
    • None

    Description

      Currently the stage table displays the file name and line number that is the call site that triggered the given stage. This is enormously useful for understanding the execution. But once a project adds utility classes and other indirections, the call site can become less meaningful, because the interesting line is further up the stack.

      An idea to fix this is to display the entire call stack that triggered the stage. It would be collapsed by default and could be revealed with a click.

      I have started working on this. It is a good way to learn about how the RDD interface ties into the UI.

      Attachments

        1. example-html.tgz
          33 kB
          Daniel Darabos

        Activity

          People

            darabos Daniel Darabos
            darabos Daniel Darabos
            Votes:
            3 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: