[HADOOP-444] In streaming with a NONE reducer, you get duplicate files if a mapper fails, is restarted, and succeeds next time.

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.5.0
• Fix Version/s: 0.7.0
• Assignee: Michel Tourn
• Reporter: Dick King
• Created: 10/Aug/06 19:22
• Updated: 08/Jul/09 17:05
• Resolved: 20/Sep/06 22:31

Description

When the dust settled after a streaming run, the directory ended up looking like this:

/user/dking/<project-name>/K-HTML-UTF8-2006-08-09-rescued-abstracted/task_0026_m_007384_0 <r 3> 10563406
/user/dking/<project-name>/K-HTML-UTF8-2006-08-09-rescued-abstracted/task_0026_m_007384_1 <r 3> 10563406

Future processing will receive duplicated data.

-dk