Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-1537 Add integration with Yarn's Application Timeline Server
  3. SPARK-11315

Add YARN extension service to publish Spark events to YARN timeline service

Attach filesAttach ScreenshotVotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Won't Fix
    • 1.5.1
    • None
    • YARN
    • None
    • Hadoop 2.6+

    Description

      Add an extension service (using SPARK-11314) to subscribe to Spark lifecycle events, batch them and forward them to the YARN Application Timeline Service. This data can then be retrieved by a new back end for the Spark History Service, and by other analytics tools.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            stevel@apache.org Steve Loughran
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Issue deployment