[HADOOP-4063] Separate spill thresholds for serialization/accounting in MapTask

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Won't Fix
• Assignee: Christopher Douglas
• Reporter: Christopher Douglas
• Created: 03/Sep/08 20:52
• Updated: 02/May/13 02:29
• Resolved: 05/Sep/08 21:32

Description

In MapTask, there is a single parameter controlling the threshold for starting a spill thread concurrently with collection. However, some users may want to set different thresholds for the serialization buffer (holding record bytes) and the accounting buffer (holding record metadata).

Issue Links

• depends upon: HADOOP-3446 The reduce task should not flush the in memory file system before starting the reducer