[SPARK-50045] Spark supports executor HostNetwork on k8s mode

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 4.0.0
• Component/s: k8s
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: zzzzming95
• Created: 3 days ago 14:29
• Updated: 3 days ago 14:45

Description

In k8s mode, when the executor executes the host network mode, it means that the executor shares the same IP address with the host. To support this mode in Spark, spark need to allow the Executor to be launched using a random BlockManager port. And prohibit shuffle local read, as this will cause the shuffle to be read failed.

Issue Links

• links to: GitHub Pull Request #48568