[SUBMARINE-224] [Umbrella] Submarine Server

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.4.0
• Component/s: Backend Server
• Target Version: 0.4.0
• Assignee: Wanqiang Ji
• Reporter: Wanqiang Ji
• Created: 09/Oct/19 06:29
• Updated: 20/Jun/20 14:23
• Resolved: 20/Jun/20 14:23

Description

This is an umbrella JIRA that covers the submarine-server module. It provides REST APIs to accept the generic job spec and manages jobs.

Design doc: https://docs.google.com/document/d/1vCjt2z3DoagfeaVQV0nfyWqSYLymoP53rDcPcz-YcSc

Issue Links

• contains: SUBMARINE-154 [Umbrella] Submarine on K8s
• is a parent of: SUBMARINE-351 [Umbrella] Support Job Management (e.g., submission) on YARN by REST APIs
• is a parent of: SUBMARINE-403 [Umbrella] Support PyTorch job management on K8s in submarine server
• is related to: SUBMARINE-98 [Umbrella] Submarine web page
• relates to: SUBMARINE-225 [Umbrella] Submarine Client
• mentioned in: Page Loading...

Sub-Tasks

• [WIP] Add submarine server architecture doc: Resolved
• [Umbrella] Submarine Job Spec: Resolved
• Initial job server which defines simple job spec and REST skelon.: Resolved
• [SS]Add k8s job submitter in submarine server to run TF job on k8s: Resolved
• [WIP] Add submitter-k8s module: Resolved
• Add travis support for K8s submitter: Resolved
• Fix failed to load class org.slf4j.impl.StaticLoggerBinder in submitter-k8s: Resolved
• Integration testing for submarine-server: Resolved
• Refactoring submarine server rest test framework: Resolved
• Add UT for SubmitterManager: Resolved
• Merge workbench-server into submarine server: Resolved
• Add JobManager and SubmitterManager components: Resolved
• Hook REST and JobManager to accept TFJob: Resolved
• Fix unchecked call 'result<T>' when used JsonResponse.Builder build response: Resolved
• Pack the k8s submitter libs into lib/submitter/k8s: Resolved
• Fix submarine server database connection failure: Resolved
• Docs of submarine server deployment: Resolved
• Support get job's log in submarine-server REST API: Resolved
• Support find/patch/delete job by REST: Resolved
• Submarine server should support helm charts installation: Resolved
• Simplify the submitter configuration: Resolved