[SPARK-374] Add a limit on the number of parallel fetches in the reduce stage

• Type: Bug
• Status: Resolved
• Resolution: Fixed
• Assignee: Unassigned
• Reporter: Harvey Feng
• Created: 27/Aug/12 15:49
• Updated: 19/Oct/12 22:50
• Resolved: 19/Oct/12 22:50

Description

Made the numer of parallel fetches per reduce task
System.getProperty("spark.default.parallelism", "8")
by default, but it can be set by setting the 'spark.blockManager.parallelFetches' property.