[MAPREDUCE-1776] Hadoop should provide a simplified way of fetching user-logs

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 0.20.1, 0.20.2
• Component/s: tasktracker
• Tags: LOGS, HADOOP, FETCHING
• Assignee: Unassigned
• Reporter: Alok Singh
• Created: 20/Mar/10 04:08
• Updated: 30/Jul/14 16:41
• Resolved: 30/Jul/14 16:41

Description

Hi,

Currently , in both streaming and normal mode hadoop mapred program, user can't download the
logs programatically.

jobclient.output.filter=ALL, allows user to print output in the stdout/stderr

We need to have the ability to fetch the user logs from the framework.

in 0.18, under the hod , we used to store the logs in the hdfs.
For 0.20, as long as the jobhistory resides user can view the logs but then it's gone.

So in short , I am proposing that hadoop
1) provide the options to the JobClient (i.e jobconf var jobclient.output.logs.location), so that it will download the output in the hdfs location instead of printing to stdout.

Thanks
Alok

Alok