Uploaded image for project: 'Apache Nemo'
  1. Apache Nemo
  2. NEMO-20

RESTful APIs to Access Job State and Metric

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

      Description

      We should provide RESTful APIs to access various job states and metric (ex. execution times) for NEMO users.

      by Yunseong Lee
      I'm implementing the HTTP server on my branch 528-rest-api.

      The description of APIs can be found at https://goo.gl/4ngKBb; currently job-state and executors are implemented.

      You can test the feature by simply running an example and by sending HTTP requests as follows:

      $ curl "http://localhost:19614/vortex/v1/executors"
      {"running": [Executor-2, Executor-3, Executor-1], "failed": []}
      $ curl "http://localhost:19614/vortex/v1/job-state"
      {"dag": {"vertices": [{"id": "Stage-0", "properties": {"scheduleGroupIndex": 0, "taskGroupList": [{"taskGroupId": "TaskGroup-1", "taskGroupIdx": 0, "taskDAG": {"vertices": [{"id": "Task-1", "properties": {"runtimeVertexId": "vertex1", "index": 0}}, {"id": "Task-2", "properties": {"runtimeVertexId": "vertex5", "index": 0}}],
       "edges": [{"src": "Task-1", "dst": "Task-2", "properties": {"runtimeEdgeId": "edge4", "edgeProperties": {"DataCommunicationPattern": "class edu.snu.vortex.runtime.executor.datatransfer.data_communication_pattern.OneToOne", "DataFlowModel": "Pull", "DataStore": "class edu.snu.vortex.runtime.executor.data.MemoryStore", "Partitioning": "class edu.snu.vortex.runtime.executor.datatransfer.partitioning.Hash"}, "coder": "StringUtf8Coder"}}]}, "containerType": "None"}]}},
      ...
      

      NOTE: For now, you need to find the HTTP server's port number from the `driver.stderr` log, but we can set `TcpPortRangeBegin` with which Driver starts scanning available ports.

        Attachments

        Issue Links

          Activity

            People

            • Assignee:
              skystar Jae Hyeon Park
              Reporter:
              sanha Sanha Lee

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment