[FLINK-29281] Replace Akka by gRPC-based RPC implementation

• Type: Technical Debt
• Status: In Progress
• Priority: Major
• Resolution: Unresolved
• Component/s: Runtime / RPC
• Assignee: Chesnay Schepler
• Reporter: Chesnay Schepler
• Created: 13/Sep/22 11:14
• Updated: 27/Jul/23 08:09

Description

Following the license change I propose to eventually replace Akka.

Based on LEGAL-619 an exemption is not feasible, and while a fork may be created it's long-term future is up in the air and I'd be uncomfortable with relying on it.

I've been experimenting with a new RPC implementation based on gRPC and so far I'm quite optimistic. It's also based on Netty while not requiring as much of a tight coupling as Akka did.
This would also allow us to sidestep migrating our current Akka setup from Netty 3 (which is affected by several CVEs) to Akka Artery, both saving work and not introducing an entirely different network stack to the project.

Issue Links

• relates to: FLINK-32468 Replace Akka by Pekko
• mentioned in: Page Loading...

Sub-Tasks

• RpcConnectionTest may pass invalid rpc URL: Closed
• Move getSelfGateway into RpcService: Closed
• ResourceManagerTest#testProcessResourceRequirementsWhenRecoveryFinished prone to race condition: Closed
• Make RPC timeout extraction reusable: Closed
• RpcGateway should have some form of Close() method: Reopened
• Move common RPC utils to rpc-core: Closed