[HADOOP-318] Progress in writing a DFS file does not count towards Job progress and can make the task timeout

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.3.2
• Fix Version/s: 0.4.0

Environment

all, but especially on big busy clusters

• Assignee: Milind Barve
• Reporter: Milind Barve
• Created: 23/Jun/06 03:14
• Updated: 08/Jul/09 16:51
• Resolved: 29/Jun/06 04:04

Description

When a task writes to DFS file, depending on how busy the cluster is, it can timeout after 10 minutes by default, because the progress towards writing a DFS file does not count as progress of the task. The solution (patch is forthcoming) is to provide a way to callback reporter to report task progress from DFSOutputStream.