[MAPREDUCE-5736] Jobtracker to hang when jobs with lot of tasks running

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Component/s: jobtracker
• Assignee: Unassigned
• Reporter: Benoit Perroud
• Created: 27/Jan/14 09:13
• Updated: 27/Jan/14 17:32

Description

The jobtracker (in MRv1) is progressing slowly when a job with lot of tasks is running. The reason is that JT.getJobCounters is holding a global lock, and with a big job (like 50+K mappers for instance), it could take while to instanciate the ``Counters`` class. This global lock prevent all other activities to run normally, queuing them and degrading the normal activities of the JT.

I was wondering if job.getCounters(), which is synchronized on a finer granularity (i.e. per job and not global) couldn't be taken out of the synchronized block.

On a more general idea, I was wondering if the usage of the synchronized statement in the JT shouldn't be re-thought. Or maybe all this has already been addressed in YARN.