[#HADOOP-4906] TaskTracker running out of memory after running several tasks

Hadoop Common

TaskTracker running out of memory after running several tasks

Created: 17/Dec/08 07:15 PM Updated: 08/Jul/09 04:53 PM

Return to search

Component/s:

None

Affects Version/s:

0.19.0

Fix Version/s:

0.19.1

Time Tracking:

Not Specified

File Attachments:

File Name	Date Attached	Attached By	Size
4906_v1.patch	2008-12-23 01:01 PM	Sharad Agarwal	1 kB
4906_v2.patch	2008-12-29 02:06 PM	Sharad Agarwal	4 kB
4906_v3.patch	2009-01-05 11:26 AM	Sharad Agarwal	4 kB
4906_v4.patch	2009-01-12 12:40 AM	Sharad Agarwal	1.0 kB
4906_v5.patch	2009-01-15 01:32 AM	Sharad Agarwal	2 kB

Release Note:	Fix the tasktracker for OOM exception by sharing the jobconf properties across tasks of the same job. Earlier a new instance was held for each task. With this fix, the job level configuration properties are shared across tasks of the same job.
Resolution Date:	16/Jan/09 12:35 AM

Description

« Hide

Looks like the TaskTracker isn't cleaning up correctly after completed/failed tasks, I suspect that the JobConfs aren't being deallocated. Eventually the TaskTracker runs out of memory after running several tasks.

All

Comments

Work Log

Change History

Subversion Commits

Sort Order:

Repository	Revision	Date	User	Message
ASF	#734870	Fri Jan 16 00:30:26 UTC 2009	acmurthy	~~HADOOP-4906~~. Fix TaskTracker OOM by keeping a shallow copy of JobConf in TaskTracker.TaskInProgress. Contributed by Sharad Agarwal.
				Files Changed
				MODIFY /hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/TaskTracker.java MODIFY /hadoop/core/trunk/CHANGES.txt

Repository	Revision	Date	User	Message
ASF	#734871	Fri Jan 16 00:32:26 UTC 2009	acmurthy	Merge -r 734869:734870 from trunk to BRANCH-0.20 to fix ~~HADOOP-4906~~.
				Files Changed
				MODIFY /hadoop/core/branches/branch-0.20/CHANGES.txt MODIFY /hadoop/core/branches/branch-0.20/src/mapred/org/apache/hadoop/mapred/TaskTracker.java MODIFY /hadoop/core/branches/branch-0.20

Repository	Revision	Date	User	Message
ASF	#734873	Fri Jan 16 00:34:41 UTC 2009	acmurthy	Merge -r 734869:734870 from trunk to BRANCH-0.19 to fix ~~HADOOP-4906~~.
				Files Changed
				MODIFY /hadoop/core/branches/branch-0.19 MODIFY /hadoop/core/branches/branch-0.19/src/mapred/org/apache/hadoop/mapred/TaskTracker.java MODIFY /hadoop/core/branches/branch-0.19/CHANGES.txt