[BEAM-14130] Implement Jupyterlab extension for managing Dataproc clusters

• Type: New Feature
• Status: Resolved
• Priority: P2
• Resolution: Implemented
• Fix Version/s: 2.39.0
• Component/s: runner-py-interactive
• Assignee: Victor Chen
• Reporter: Victor Chen
• Created: 18/Mar/22 23:08
• Updated: 31/Mar/22 17:52
• Resolved: 31/Mar/22 17:52

Description

• Adds the 'Manage Clusters' option under the Interactive Beam JupyterLab extension to enable users to manage Interactive Beam clusters. Users are presented with the functionality to:
  • View all clusters managed by Interactive Beam.
  • Delete a selected cluster - deleting a cluster will call the ib.cleanup() routine for each pipeline associated with a given cluster 
    and clear the master_url saved in the options of the pipelines. Upon re-running the pipelines with the original pipeline options, the deleted cluster will be recreated.
  • Select a default cluster - this will make any future pipelines that do not have a Flink master_url and no specified Google Cloud project use this cluster.

Issue Links

• links to: GitHub Pull Request #17127