Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-3644

REST API for Spark application info (jobs / stages / tasks / storage info)

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.4.0
    • Component/s: Spark Core, Web UI
    • Labels:
      None

      Description

      This JIRA is a forum to draft a design proposal for a REST interface for accessing information about Spark applications, such as job / stage / task / storage status.

      There have been a number of proposals to serve JSON representations of the information displayed in Spark's web UI. Given that we might redesign the pages of the web UI (and possibly re-implement the UI as a client of a REST API), the API endpoints and their responses should be independent of what we choose to display on particular web UI pages / layouts.

      Let's start a discussion of what a good REST API would look like from first-principles. We can discuss what urls / endpoints expose access to data, how our JSON responses will be formatted, how fields will be named, how the API will be documented and tested, etc.

      Some links for inspiration:

      https://developer.github.com/v3/
      http://developer.netflix.com/docs/REST_API_Reference
      https://helloreverb.com/developers/swagger

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                imranr Imran Rashid
                Reporter:
                joshrosen Josh Rosen
              • Votes:
                7 Vote for this issue
                Watchers:
                26 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: