[SPARK-44920] Use await() instead of awaitUninterruptibly() in TransportClientFactory.createClient()

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.3, 3.4.2, 3.5.0
• Fix Version/s: 3.4.2, 3.5.0, 4.0.0, 3.3.4
• Component/s: Spark Core
• Assignee: Josh Rosen
• Reporter: Josh Rosen
• Created: 23/Aug/23 00:40
• Updated: 23/Aug/23 05:59
• Resolved: 23/Aug/23 05:55

Description

This is a follow up for SPARK-44241:

That call added an `awaitUninterruptibly()` call, which I think should be a plain `await()` instead. This will prevent issues when cancelling tasks with hanging network connections. 

This issue is similar to SPARK-19529

Issue Links

• is caused by: SPARK-44241 Set io.connectionTimeout/connectionCreationTimeout to zero or negative will cause executor incessantes cons/destructions
• relates to: SPARK-19529 TransportClientFactory.createClient() shouldn't call awaitUninterruptibly()
• links to: [Github] Pull Request #42619 (JoshRosen)