[MAPREDUCE-7017] Too many times of meaningless invocation in TaskAttemptImpl#resolveHosts

• Type: Improvement
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.0.0-alpha4
• Component/s: mr-am
• Labels: pull-request-available
• Assignee: jiayuhan-it
• Reporter: jiayuhan-it
• Created: 04/Dec/17 08:13
• Updated: 23/Dec/20 04:47

Description

MRAppMaster uses TaskAttemptImpl::resolveHosts to determine the dataLocalHosts for each task when the location of data split is IP, which will call a lot of times ( taskNum * dfsReplication) of function InetAddress::getByName and most of the funcition calls are redundant. When the job has a great number of tasks and the speed of DNS resolution is not fast enough, it will take a lot of time at this stage before the job running.

Issue Links

• links to: GitHub Pull Request #309
• links to: GitHub Pull Request #309