[PIG-3913] Pig should use job's jobClient wherever possible (fixes local mode counters)

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.13.0
• Fix Version/s: 0.13.0
• Assignee: Aniket Namadeo Mokashi
• Reporter: Aniket Namadeo Mokashi
• Created: 30/Apr/14 00:13
• Updated: 07/Jul/14 18:08
• Resolved: 20/May/14 17:50

Description

MapreduceLauncher initializes a statsJobClient to poll counter information of jobs. This works fine in mapreduce mode but it reports incorrect information in local (auto-local) mode. Pig code should try to use org.apache.hadoop.mapred.jobcontrol.Job's getJobClient api to get handle to jobClient wherever possible. statsJobClient (and wherever its references are passed) should be deprecated.

Issue Links

• breaks: PIG-3958 TestMRJobStats is broken in 0.13 and trunk
• is related to: PIG-3853 Pig does not kill all the jobs during shutdown for hadoop2
• is related to: PIG-3825 Stats collection needs to be changed for hadoop2 (with auto local mode)
• is related to: MAPREDUCE-1214 Add support for counters in Hadoop Local Mode