[MAPREDUCE-605] In Streaming, allow different mappers for different subsets of the input

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: contrib/streaming
• Assignee: Unassigned
• Reporter: arkady borkovsky
• Created: 26/Nov/07 17:29
• Updated: 02/May/13 02:29

Description

The command line may look like this:

-mapper mapper-command-1 -input dir11 -mapper mapper-command-2 -input dir22 input -dir21

meaning that map phase will apply mapper-command-1 to part files from dir11, and the part files from dir22 and dir21 will be processed by mapper-command-2
then all will be shuffled and processed by a single reducer.

Issue Links

• depends upon: HADOOP-372 should allow to specify different inputformat classes for different input dirs for Map/Reduce jobs