[SPARK-3137] Use finer grained locking in TorrentBroadcast.readObject

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0
• Component/s: Spark Core
• Assignee: Shixiong Zhu
• Reporter: Reynold Xin
• Created: 19/Aug/14 20:33
• Updated: 03/Sep/19 21:13
• Resolved: 03/Sep/19 21:13

Description

TorrentBroadcast.readObject uses a global lock so only one task can be fetching the blocks at the same time.

This is not optimal if we are running multiple stages concurrently because they should be able to independently fetch their own blocks.

Issue Links

• links to: GitHub Pull Request #25612