[TEZ-696] Remove implicit copying of processor payload to input and output

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.5.0
• Hadoop Flags: Incompatible change, Reviewed
• Assignee: Bikas Saha
• Reporter: Bikas Saha
• Created: 29/Dec/13 13:03
• Updated: 06/Sep/14 01:35
• Resolved: 16/Jul/14 17:27

Description

In the task, the processor payload is implicitly copied to the input and output if they dont have one. While this may have been convenient in some cases, its very confusing when things dont work or when they surprisingly work. Also, its probably clearer to specify payload where appropriate so that its clear which one is for which. While writing the word count example I had to spend a lot of time debugging why my outputs had stopped working when I removed the map reduce confs from my non-MR processors.

Issue Links

• blocks: TEZ-872 Both tez.runtime.intermediate-output.should-compress and tez.runtime.intermediate-input.is-compressed seem unnecessary
• is blocked by: HIVE-6483 Hive on Tez - Hive should create different payloads for inputs and outputs