[#HADOOP-3848] TaskTracker.localizeJob calls getSystemDir for each task rather than caching it

Hadoop Common

TaskTracker.localizeJob calls getSystemDir for each task rather than caching it

Created: 29/Jul/08 04:44 AM Updated: 08/Jul/09 04:52 PM

Return to search

Component/s:

None

Affects Version/s:

0.18.0

Fix Version/s:

0.19.0

Time Tracking:

Not Specified

File Attachments:

	File Name	Date Attached	Attached By	Size
	HADOOP-3848_0_20080729.patch	2008-07-29 08:13 PM	Arun C Murthy	3 kB

Hadoop Flags:	Reviewed
Resolution Date:	30/Jul/08 10:16 PM

Description

« Hide

We should call it once and cache that value, I see 100k+ RPCs for large jobs...

All

Comments

Work Log

Change History

Subversion Commits

Sort Order:

Arun C Murthy made changes - 29/Jul/08 08:13 PM

Field	Original Value	New Value
Attachment		HADOOP-3848_0_20080729.patch [ 12387129 ]

Arun C Murthy made changes - 29/Jul/08 08:13 PM

Status

Open [ 1 ]

Patch Available [ 10002 ]

[ Permlink | « Hide ]

Hadoop QA added a comment - 29/Jul/08 11:14 PM

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12387129/HADOOP-3848_0_20080729.patch
against trunk revision 680823.

+1 @author. The patch does not contain any @author tags.

-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac compiler warnings.

+1 findbugs. The patch does not introduce any new Findbugs warnings.

+1 release audit. The applied patch does not increase the total number of release audit warnings.

+1 core tests. The patch passed core unit tests.

+1 contrib tests. The patch passed contrib unit tests.

Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2975/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2975/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2975/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2975/console

This message is automatically generated.

[ Permlink | « Hide ]

Amareshwari Sriramadasu added a comment - 30/Jul/08 06:48 AM

+1
Patch looks good.

Repository	Revision	Date	User	Message
ASF	#681223	Wed Jul 30 22:16:45 UTC 2008	omalley	~~HADOOP-3848~~. Cache calls to getSystemDir in the TaskTracker instead of calling it for each task start. Contributed by Arun C. Murthy.
				Files Changed
				MODIFY /hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/TaskTracker.java MODIFY /hadoop/core/trunk/CHANGES.txt

[ Permlink | « Hide ]

Owen O'Malley added a comment - 30/Jul/08 10:16 PM

I just committed this. Thanks, Arun!

Owen O'Malley made changes - 30/Jul/08 10:16 PM

Resolution		Fixed [ 1 ]
Hadoop Flags		[Reviewed]
Status	Patch Available [ 10002 ]	Resolved [ 5 ]

[ Permlink | « Hide ]

Hudson added a comment - 22/Aug/08 12:34 PM

Integrated in Hadoop-trunk #581 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/581/)

Nigel Daley made changes - 20/Nov/08 11:38 PM

Status

Resolved [ 5 ]

Closed [ 6 ]

Owen O'Malley made changes - 08/Jul/09 04:52 PM

Component/s

mapred [ 12310690 ]