[SUBMARINE-725] [Umbrella] Support running qlib on Submarine

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.6.0
• Component/s: experiment
• Target Version: 0.6.0
• Assignee: Kevin Su
• Reporter: Kevin Su
• Created: 01/Feb/21 09:08
• Updated: 28/Aug/21 07:19
• Resolved: 28/Aug/21 07:19

Description

We plan to integrate qlib into the submarine after discussion in a weekly online meeting.

The goal of this Jira is

• Run qlib workflow in the notebook
• Run qlib workflow in the experiment (Both single node and multi-node training)
• Support MLflow UI on the workbench (Launch a pod in K8s like tensorboard) since qlib will write underly metadata (metrics and parameters) to mlflow server
• Support different versions of the qlib library in the environment for users to choose from.

Sub-Tasks

• Run qlib workflow in the notebook: Resolved
• Support MLflow server on Submarine: Resolved
• Frontend support for MLflow UI: Resolved
• Support MLflow model registry: Resolved