It would be useful to track 'first-task-launched' time to job-summary for better reporting.
Common portion of MAPREDUCE-1545
Looks like it's not committed to the y20s branch yet. Updated the y20s branch patch and committing.
I committed this, eliding the whitespace change to TaskInProgress. Thanks, Luke!
@ciemo, you can find start and finish times of every task in job history. The first task launch times are for the job summary only.
Is this only for "first task" or are we measuring wait times in pending state for ALL tasks?
Last week, we ran out of reduce slots. I had one job where 150 of my reduce tasks had completed but other 150 were in a pending state (not running) for over 1 hour.
Are we measuring pending time for all tasks or only for first task?
-1 overall. Here are the results of testing the latest attachment
against trunk revision 937201.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/134/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/134/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/134/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/134/console
This message is automatically generated.
Integrated in Hadoop-Common-trunk-Commit #221 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk-Commit/221/)
HADOOP-6657. Add a capitalization method to StringUtils for MAPREDUCE-1545
v2 trunk patch added tests to assert precise launch times.
v1 patch for trunk as well.
v3 patch for y20 branch to incorporate feedbacks from HADOOP-6657
Patch for yahoo 0.20.1xx (y20s) branch.
Hong Tang mentioned that first tasks are setup tasks and we'd better add firstMapTaskLaunchTime and firstReduceTaskLaunchTime besides the firstTaskLaunchTime while we're at it.
Our reporting team wants this for better wait time calculations in our reporting DB.