[MAPREDUCE-2264] Job status exceeds 100% in some cases

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.20.2, 0.20.205.0
• Fix Version/s: 1.2.0, 2.0.3-alpha
• Component/s: jobtracker
• Labels: critical-0.22.0
• Target Version/s: 0.22.0, 1.2.0
• Hadoop Flags: Reviewed
• Assignee: Devaraj Kavali
• Reporter: Adam Kramer
• Created: 14/Jan/11 02:27
• Updated: 17/Jul/14 17:55
• Resolved: 29/Jan/13 19:40

Description

I'm looking now at my jobtracker's list of running reduce tasks. One of them is 120.05% complete, the other is 107.28% complete.

I understand that these numbers are estimates, but there is no case in which an estimate of 100% for a non-complete task is better than an estimate of 99.99%, nor is there any case in which an estimate greater than 100% is valid.

I suggest that whatever logic is computing these set 99.99% as a hard maximum.

Issue Links

• duplicates: MAPREDUCE-3267 MR2 reduce tasks showing >100% complete
• duplicates: MAPREDUCE-428 Reducers reported completion % is generally incorrect when consuming compressed map outputs
• is related to: MAPREDUCE-4965 In merge, no ordering defined for CompressAwarePath
• is related to: HADOOP-5210 Reduce Task Progress shows > 100% when the total size of map outputs (for a single reducer) is high
• is related to: HADOOP-5572 The map progress value should have a separate phase for doing the final sort.