[MAPREDUCE-2524] Backport trunk heuristics for failing maps when we get fetch failures retrieving map output during shuffle

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.20.204.0
• Fix Version/s: 0.20.204.0
• Component/s: tasktracker
• Hadoop Flags: Reviewed
• Release Note: Added a new configuration option: mapreduce.reduce.shuffle.maxfetchfailures, and removed a no longer used option: mapred.reduce.copy.backoff.
• Assignee: Thomas Graves
• Reporter: Thomas Graves
• Created: 20/May/11 14:06
• Updated: 02/Sep/11 22:13
• Resolved: 04/Jun/11 01:00

Description

The heuristics for failing maps when we get map output fetch failures during the shuffle is pretty conservative in 20. Backport the heuristics from trunk which are more aggressive, simpler, and configurable.