[#HADOOP-4966] Setup tasks are not removed from JobTracker's taskIdToTIPMap even after the job completes

Hadoop Common

Setup tasks are not removed from JobTracker's taskIdToTIPMap even after the job completes

Created: 31/Dec/08 10:37 AM Updated: 08/Jul/09 04:53 PM

Return to search

Component/s:

None

Affects Version/s:

0.20.0

Fix Version/s:

0.19.1

Time Tracking:

Not Specified

File Attachments:

	File Name	Date Attached	Attached By	Size
	patch-4966.txt	2009-01-02 04:48 AM	Amareshwari Sriramadasu	1 kB

Issue Links:

dependent

This issue is depended upon by:

MAPREDUCE-177 Hadoop performance degrades significantly as more and more jobs complete

Hadoop Flags:	Reviewed
Resolution Date:	08/Jan/09 05:54 AM

Description

« Hide

When a task is launched, a TaskID to TaskInProgress mapping is added to taskidToTIPMap in JobTracker. Upon task completion the mapping is removed. Mapping for all the tasks are removed (including cleanup tasks) except setup tasks.

All

Comments

Work Log

Change History

Subversion Commits

Sort Order:

Amar Kamat made changes - 31/Dec/08 10:38 AM

Field	Original Value	New Value
Link		This issue is depended upon by HADOOP-4766 [ HADOOP-4766 ]

Amar Kamat made changes - 02/Jan/09 04:17 AM

Affects Version/s	0.21.0 [ 12313563 ]
Affects Version/s		0.20.0 [ 12313438 ]
Fix Version/s	0.21.0 [ 12313563 ]
Fix Version/s		0.20.0 [ 12313438 ]

Amareshwari Sriramadasu made changes - 02/Jan/09 04:29 AM

Assignee

Amareshwari Sriramadasu [ amareshwari ]

[ Permlink | « Hide ]

Amareshwari Sriramadasu added a comment - 02/Jan/09 04:48 AM

Patch with the fix

Amareshwari Sriramadasu made changes - 02/Jan/09 04:48 AM

Attachment

patch-4966.txt [ 12397008 ]

Amareshwari Sriramadasu made changes - 02/Jan/09 04:48 AM

Status

Open [ 1 ]

Patch Available [ 10002 ]

[ Permlink | « Hide ]

Amareshwari Sriramadasu added a comment - 02/Jan/09 01:49 PM

test-patch and all core and contrib unit tests passed

Repository	Revision	Date	User	Message
ASF	#732614	Thu Jan 08 05:50:37 UTC 2009	ddas	~~HADOOP-4966~~. Marks completed setup tasks for removal. Contributed by Amareshwari Sriramadasu.
				Files Changed
				MODIFY /hadoop/core/trunk/CHANGES.txt MODIFY /hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/JobTracker.java

Repository	Revision	Date	User	Message
ASF	#732615	Thu Jan 08 05:53:21 UTC 2009	ddas	Merge -r 732613:732614 from trunk onto 0.19 branch. Fixes ~~HADOOP-4966~~.
				Files Changed
				MODIFY /hadoop/core/branches/branch-0.19/src/mapred/org/apache/hadoop/mapred/JobTracker.java MODIFY /hadoop/core/branches/branch-0.19/CHANGES.txt

Repository	Revision	Date	User	Message
ASF	#732616	Thu Jan 08 05:54:09 UTC 2009	ddas	Merge -r 732613:732614 from trunk onto 0.20 branch. Fixes ~~HADOOP-4966~~.
				Files Changed
				MODIFY /hadoop/core/branches/branch-0.20/CHANGES.txt MODIFY /hadoop/core/branches/branch-0.20/src/mapred/org/apache/hadoop/mapred/JobTracker.java

[ Permlink | « Hide ]

Devaraj Das added a comment - 08/Jan/09 05:54 AM

I just committed this. Thanks, Amareshwari!

Devaraj Das made changes - 08/Jan/09 05:54 AM

Resolution		Fixed [ 1 ]
Fix Version/s		0.19.1 [ 12313473 ]
Hadoop Flags		[Reviewed]
Status	Patch Available [ 10002 ]	Resolved [ 5 ]

Nigel Daley made changes - 24/Feb/09 11:36 PM

Fix Version/s

0.20.0 [ 12313438 ]

Nigel Daley made changes - 24/Feb/09 11:37 PM

Status

Resolved [ 5 ]

Closed [ 6 ]

Owen O'Malley made changes - 08/Jul/09 04:53 PM

Component/s

mapred [ 12310690 ]