Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-2035

Make a stage's call stack available on the UI

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • 1.1.0
    • Web UI
    • None

    Description

      Currently the stage table displays the file name and line number that is the call site that triggered the given stage. This is enormously useful for understanding the execution. But once a project adds utility classes and other indirections, the call site can become less meaningful, because the interesting line is further up the stack.

      An idea to fix this is to display the entire call stack that triggered the stage. It would be collapsed by default and could be revealed with a click.

      I have started working on this. It is a good way to learn about how the RDD interface ties into the UI.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            darabos Daniel Darabos
            darabos Daniel Darabos
            Votes:
            3 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment