[STORM-3532] Support Kubernetes in Storm Supervisor

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.0.0, 2.2.0
• Component/s: storm-server
• Assignee: Unassigned
• Reporter: Valeriys Soloviov
• Created: 06/Nov/19 13:38
• Updated: 06/Nov/19 13:38

Description

Hello,

We are using Apache Storm 1.x and we want to migrate it to 2.x on Kubernetes. I found that we can't request resources from Kubernetes directly. I searched if there is any integration with Kubernetes: when Storm Supervisor can run workers' slots inside separate pods(I refer to the https://github.com/apache/storm/blob/master/storm-server/src/main/java/org/apache/storm/daemon/supervisor/BasicContainer.java#L852) instead of run multiple supervisors on the same K8s node.

I see the following tickets https://issues.apache.org/jira/browse/STORM-911 and https://issues.apache.org/jira/browse/STORM-1740 which could be related to my proposal.

 

I propose to add an ability to Storm Supervisor manage worker processes via Kubernetes /another API and not just on the same host.