[MAPREDUCE-5626] TaskLogServlet could not get syslog

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Won't Fix
• Affects Version/s: 1.2.1
• Fix Version/s: 1.2.1
• Labels: patch

Environment

Linux version 2.6.18-238.9.1.el5
Java(TM) SE Runtime Environment (build 1.6.0_43-b01)
hadoop-1.2.1

• Target Version/s: 1.2.1
• Release Note: modify TaskLogServlet haveTaskLog method
• Assignee: Unassigned
• Reporter: yangjun
• Created: 15/Nov/13 01:59
• Updated: 18/May/15 18:12
• Resolved: 18/May/15 18:12

Description

When multiply tasks use one jvm and generated logs.
eg.
./attempt_201211220735_0001_m_000000_0:
log.index
./attempt_201211220735_0001_m_000001_0:
log.index
./attempt_201211220735_0001_m_000002_0:
log.index stderr stdout syslog
get from http://xxxxxxxx:50060/tasklog?attemptid= attempt_201211220735_0001_m_000000_0
could get stderr,stdout,but not the others,include syslog.

see TaskLogServlet.haveTaskLog() method, not check from local && log.index, but check the original path.

resolve:
modify TaskLogServlet haveTaskLog method
private boolean haveTaskLog(TaskAttemptID taskId, boolean isCleanup,
TaskLog.LogName type) throws IOException {
File f = TaskLog.getTaskLogFile(taskId, isCleanup, type);
if (f.exists() && f.canRead())

{ return true; }

else {
File indexFile = TaskLog.getIndexFile(taskId, isCleanup);
if (!indexFile.exists())

{ return false; }

BufferedReader fis;
try

{ fis = new BufferedReader(new InputStreamReader( SecureIOUtils.openForRead(indexFile, TaskLog.obtainLogDirOwner(taskId)))); }

catch (FileNotFoundException ex) {
LOG.warn("Index file for the log of " + taskId
+ " does not exist.");

// Assume no task reuse is used and files exist on attemptdir
StringBuffer input = new StringBuffer();
input.append(LogFileDetail.LOCATION
+ TaskLog.getAttemptDir(taskId, isCleanup) + "\n");
for (LogName logName : TaskLog.LOGS_TRACKED_BY_INDEX_FILES)

{ input.append(logName + ":0 -1\n"); }

fis = new BufferedReader(new StringReader(input.toString()));
}

try {
String str = fis.readLine();
if (str == null)

{ // thefile doesn't have anything throw new IOException("Index file for the log of " + taskId + "is empty."); }

String loc = str.substring(str.indexOf(LogFileDetail.LOCATION)
+ LogFileDetail.LOCATION.length());
File tf = new File(loc, type.toString());
return tf.exists() && tf.canRead();

} finally

{ if (fis != null) fis.close(); }

}

}

workaround:
url add filter=SYSLOG could print syslog also.