[MAPREDUCE-5974] Allow specifying multiple MapOutputCollectors with fallback

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.6.0
• Fix Version/s: 2.6.0
• Component/s: task
• Target Version/s: 2.6.0
• Hadoop Flags: Reviewed
• Assignee: Todd Lipcon
• Reporter: Todd Lipcon
• Created: 17/Jul/14 16:15
• Updated: 14/Dec/14 04:22
• Resolved: 21/Aug/14 17:40

Description

Currently we only allow specifying a single MapOutputCollector implementation class in a job. It would be nice to allow a comma-separated list of classes: we should try each collector implementation in the user-specified order until we find one that can be successfully instantiated and initted.

This is useful for cases where a particular optimized collector implementation cannot operate on all key/value types, or requires native code. The cluster administrator can configure the cluster to try to use the optimized collector and fall back to the default collector.

Issue Links

• breaks: MAPREDUCE-6194 Bubble up final exception in failures during creation of output collectors