Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-4440

Enhance the job progress API to expose more information

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Incomplete
    • None
    • None
    • Spark Core

    Description

      The progress API introduced in SPARK-2321 provides a new way for user to monitor job progress. However the information exposed in the API is relatively limited. It'll be much more useful if we can enhance the API to expose more data.
      Some improvement for example may include but not limited to:
      1. Stage submission and completion time.
      2. Task metrics.
      The requirement is initially identified for the hive on spark project(HIVE-7292), other application should benefit as well.

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            lirui Rui Li
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Issue deployment