[NEMO-20] RESTful APIs to Access Job State and Metric

• Type: New Feature
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 0.1
• Labels: client gsoc2018 pull-request-available runtime
• Assignee: Jae Hyeon Park
• Reporter: Sanha Lee
• Created: 02/Mar/18 11:12
• Updated: 27/Nov/18 03:42
• Resolved: 20/Jul/18 03:01

Description

We should provide RESTful APIs to access various job states and metric (ex. execution times) for NEMO users.

by yunseong
I'm implementing the HTTP server on my branch 528-rest-api.

The description of APIs can be found at https://goo.gl/4ngKBb; currently job-state and executors are implemented.

You can test the feature by simply running an example and by sending HTTP requests as follows:

$ curl "http://localhost:19614/vortex/v1/executors"
{"running": [Executor-2, Executor-3, Executor-1], "failed": []}
$ curl "http://localhost:19614/vortex/v1/job-state"
{"dag": {"vertices": [{"id": "Stage-0", "properties": {"scheduleGroupIndex": 0, "taskGroupList": [{"taskGroupId": "TaskGroup-1", "taskGroupIdx": 0, "taskDAG": {"vertices": [{"id": "Task-1", "properties": {"runtimeVertexId": "vertex1", "index": 0}}, {"id": "Task-2", "properties": {"runtimeVertexId": "vertex5", "index": 0}}],
 "edges": [{"src": "Task-1", "dst": "Task-2", "properties": {"runtimeEdgeId": "edge4", "edgeProperties": {"DataCommunicationPattern": "class edu.snu.vortex.runtime.executor.datatransfer.data_communication_pattern.OneToOne", "DataFlowModel": "Pull", "DataStore": "class edu.snu.vortex.runtime.executor.data.MemoryStore", "Partitioning": "class edu.snu.vortex.runtime.executor.datatransfer.partitioning.Hash"}, "coder": "StringUtf8Coder"}}]}, "containerType": "None"}]}},
...

NOTE: For now, you need to find the HTTP server's port number from the `driver.stderr` log, but we can set `TcpPortRangeBegin` with which Driver starts scanning available ports.

Issue Links

• blocks: NEMO-25 Improve WebUI to use RESTful APIs by Nemo
• blocks: NEMO-159 Nemo Web UI
• links to: GitHub Pull Request #61