[SPARK-1775] Unneeded lock in ShuffleMapTask.deserializeInfo

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 0.9.0, 0.9.1, 1.0.0
• Fix Version/s: 0.9.2, 1.0.0
• Component/s: Spark Core
• Labels: Starter
• Assignee: Sandeep Singh
• Reporter: Matei Alexandru Zaharia
• Created: 09/May/14 02:06
• Updated: 10/May/14 06:11
• Resolved: 09/May/14 05:31

Description

This was used in the past to have a cache of deserialized ShuffleMapTasks, but that's been removed, so there's no need for a lock. It slows down Spark when task descriptions are large, e.g. due to large lineage graphs or local variables.