[MAPREDUCE-1362] Pipes should be ported to the new mapreduce API

• Type: Improvement
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Component/s: pipes
• Release Note: Update Hadoop Pipes to support MRv2 API
• Assignee: Unassigned
• Reporter: Bassam Tabbara
• Created: 07/Jan/10 08:04
• Updated: 06/Nov/15 02:43

Description

Pipes is still currently using the old mapred API. This prevents us from using pipes with HBase's TableInputFormat, HRegionPartitioner, etc.

Here is a rough proposal for how to accomplish this:

• Add a new package org.apache.hadoop.mapreduce.pipes that uses the new mapred API.
• the new pipes package will run side by side with the old one. old one should get deprecated at some point.
• the wire protocol used between PipesMapper and PipesReducer and C++ programs must not change.
• bin/hadoop should support both pipes (old api) and pipes2 (new api)

Does this sound reasonable?