Description
While running GridMixV3 with high memory reduces, we ran into issues where for jobs with significant number of maps and reduces, when the map progress hits 98-99% but still there are maps pending, reduces get every new container that RM allocates. And the job takes much longer time than with usual reduces.
For addressing precisely these issues, a configurable limit was introduced to limit the reduce ramp up. Unfortunately this limit is not working correctly.
Attachments
Attachments
Issue Links
- is related to
-
MAPREDUCE-3719 Make gridmix performance on YARN+MR to match or exceed that on 1.0
-
- Resolved
-