[SPARK-42173] IPv6 address mapping can fail with sparse addresses

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0
• Component/s: Spark Core

Environment

I've only observed this on Kube but it might be able to happen on YARN since the mapping is in core.

• Assignee: Holden Karau
• Reporter: Holden Karau
• Created: 24/Jan/23 20:11
• Updated: 26/Jan/23 17:14
• Resolved: 26/Jan/23 17:13

Description

e.g. the `addressToExecutorId` ends up storing `2602:fcb1::1337:12` but when it comes time to handle the `onDisconnect` all of the implicit zeros are expanded out.

 

 

Issue Links

• links to: [Github] Pull Request #39728 (holdenk)
• links to: [Github] Pull Request #39728 (holdenk)