Uploaded image for project: 'Apache Nemo'
  1. Apache Nemo
  2. NEMO-20

RESTful APIs to Access Job State and Metric

    XMLWordPrintableJSON

    Details

      Description

      We should provide RESTful APIs to access various job states and metric (ex. execution times) for NEMO users.

      by Yunseong Lee
      I'm implementing the HTTP server on my branch 528-rest-api.

      The description of APIs can be found at https://goo.gl/4ngKBb; currently job-state and executors are implemented.

      You can test the feature by simply running an example and by sending HTTP requests as follows:

      $ curl "http://localhost:19614/vortex/v1/executors"
      {"running": [Executor-2, Executor-3, Executor-1], "failed": []}
      $ curl "http://localhost:19614/vortex/v1/job-state"
      {"dag": {"vertices": [{"id": "Stage-0", "properties": {"scheduleGroupIndex": 0, "taskGroupList": [{"taskGroupId": "TaskGroup-1", "taskGroupIdx": 0, "taskDAG": {"vertices": [{"id": "Task-1", "properties": {"runtimeVertexId": "vertex1", "index": 0}}, {"id": "Task-2", "properties": {"runtimeVertexId": "vertex5", "index": 0}}],
       "edges": [{"src": "Task-1", "dst": "Task-2", "properties": {"runtimeEdgeId": "edge4", "edgeProperties": {"DataCommunicationPattern": "class edu.snu.vortex.runtime.executor.datatransfer.data_communication_pattern.OneToOne", "DataFlowModel": "Pull", "DataStore": "class edu.snu.vortex.runtime.executor.data.MemoryStore", "Partitioning": "class edu.snu.vortex.runtime.executor.datatransfer.partitioning.Hash"}, "coder": "StringUtf8Coder"}}]}, "containerType": "None"}]}},
      ...
      

      NOTE: For now, you need to find the HTTP server's port number from the `driver.stderr` log, but we can set `TcpPortRangeBegin` with which Driver starts scanning available ports.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                skystar Jae Hyeon Park
                Reporter:
                sanha Sanha Lee
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: