|
[
Permlink
| « Hide
]
Jothi Padmanabhan added a comment - 10/Feb/09 07:37 AM
Screen shot showing progress > 100%
This could be because of the way we compute mergeProgress during merges in the reduce. The mergeProgress is a function of the totalBytesProcessed and the totalBytesProcessed is incremented for every segment considered during merge. So if we have multi-level merges, we would run into a case where we report more progress per byte since many bytes would make hit the disk but they would be again considered for the next level merge and so on..
As Devaraj mentioned, the problem is in the calculation of mergeProgress when multi-level merges happen.
Attaching patch that fixes the issue. Please review and provide your comments. Carrying over bytes from intermediate merges to the final reduce phase is not correct.
Yes Jothi. When the intermediate merges complete, we can say that the sortPhase is completed and if we reset the variable totalBytesProcessed before the final merge, we can use that for calculating the progress of reducePhase(the 3rd phase of reduce task). Patch of
Matei, Would you please check if your patch(of JIRA 3131) removed this reset intentionally and if I am missing out something ? Attaching patch which resets the bytes-processed to zero before final merge. Jothi offline suggested to remove some unnecessary code from merge().
Attaching new patch with that change. TestReduceTask was failing with earlier patch because of ignoring the starting bytes read from segments in the final merge.
Attaching the patch that resets totalBytesProcessed to the number of bytes read in this final merge(instead of 0). Please review and provide your comments. -1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12402622/HADOOP-5210.v3.patch  against trunk revision 756352. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 Eclipse classpath. The patch retains Eclipse classpath integrity. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/114/testReport/ This message is automatically generated. I just committed this. Thanks, Ravi!
Integrated in Hadoop-trunk #790 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/790/
). Solves a problem in the progress report of the reduce task. Contributed by Ravi Gummadi. It would be nice if this patch can be committed to branch 0.20 also.
The same patch applies to branch 0.20 also. Devaraj, Would you please commit this to 0.20 ? I committed this to the 0.20 branch.
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
 |
|
Bug
Resolved
Minor
Reduce Task Progress shows > 100% when the total size of map outputs (for a single reducer) is high 
