[#HADOOP-3875] Fix TaskTracker's heartbeat timer to note the time the hearbeat RPC returned to decide next heartbeat time

Hadoop Common

Fix TaskTracker's heartbeat timer to note the time the hearbeat RPC returned to decide next heartbeat time

Created: 31/Jul/08 01:19 AM Updated: 08/Jul/09 04:52 PM

Return to search

Component/s:

None

Affects Version/s:

0.18.0

Fix Version/s:

0.19.0

Time Tracking:

Not Specified

File Attachments:

	File Name	Date Attached	Attached By	Size
	HADOOP-3875_0_20080730.patch	2008-07-31 02:08 AM	Arun C Murthy	2 kB

Hadoop Flags:	Reviewed
Resolution Date:	08/Aug/08 11:36 PM

Description

« Hide

The tasktracker notes the time the heartbeat RPC was fired, rather than when it returned to decide when to send the next one. If the RPC spends a lot of time in the JobTracker's queue then the above exacerbates the load on the JobTracker.

All

Comments

Work Log

Change History

Subversion Commits

Sort Order:

[ Permlink | « Hide ]

Arun C Murthy added a comment - 31/Jul/08 01:28 AM

Also, there is a redundant call to System.currentTime in TaskTracker.transmitHeartbeat - that information is already available at TaskTracker.offerService, we should just pass it down.

[ Permlink | « Hide ]

Arun C Murthy added a comment - 31/Jul/08 02:08 AM

Straight-forward patch...

Arun C Murthy made changes - 31/Jul/08 02:08 AM

Field	Original Value	New Value
Attachment		HADOOP-3875_0_20080730.patch [ 12387250 ]

Arun C Murthy made changes - 31/Jul/08 02:15 AM

Status

Open [ 1 ]

Patch Available [ 10002 ]

[ Permlink | « Hide ]

Amareshwari Sriramadasu added a comment - 31/Jul/08 04:55 AM

Amareshwari Sriramadasu made changes - 31/Jul/08 04:55 AM

Hadoop Flags

[Reviewed]

[ Permlink | « Hide ]

Hadoop QA added a comment - 31/Jul/08 04:58 AM

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12387250/HADOOP-3875_0_20080730.patch
against trunk revision 681243.

+1 @author. The patch does not contain any @author tags.

-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 javac. The applied patch does not increase the total number of javac compiler warnings.

+1 findbugs. The patch does not introduce any new Findbugs warnings.

+1 release audit. The applied patch does not increase the total number of release audit warnings.

-1 core tests. The patch failed core unit tests.

+1 contrib tests. The patch passed contrib unit tests.

Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2990/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2990/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2990/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2990/console

This message is automatically generated.

Repository	Revision	Date	User	Message
ASF	#684142	Fri Aug 08 23:35:46 UTC 2008	omalley	~~HADOOP-3875~~. Change the time period between heartbeats to be relative to the end of the heartbeat rpc, rather than the start. This causes better behavior if the JobTracker is overloaded. (acmurthy via omalley)
				Files Changed
				MODIFY /hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/TaskTracker.java MODIFY /hadoop/core/trunk/CHANGES.txt

[ Permlink | « Hide ]

Owen O'Malley added a comment - 08/Aug/08 11:36 PM

I just committed this. Thanks, Arun!

Owen O'Malley made changes - 08/Aug/08 11:36 PM

Resolution		Fixed [ 1 ]
Status	Patch Available [ 10002 ]	Resolved [ 5 ]

[ Permlink | « Hide ]

Hudson added a comment - 22/Aug/08 12:34 PM

Integrated in Hadoop-trunk #581 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/581/)

Nigel Daley made changes - 20/Nov/08 11:38 PM

Status

Resolved [ 5 ]

Closed [ 6 ]

Owen O'Malley made changes - 08/Jul/09 04:52 PM

Component/s

mapred [ 12310690 ]