[MAPREDUCE-6720] Inconsistent values of counters across tasks and job reported to timeline service.

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: YARN-2928
• Fix Version/s: 2.9.0, 3.0.0-alpha1
• Component/s: applicationmaster
• Labels: yarn-2928-1st-milestone
• Target Version/s: YARN-2928
• Hadoop Flags: Reviewed
• Assignee: Varun Saxena
• Reporter: Varun Saxena
• Created: 20/Jun/16 19:24
• Updated: 21/Oct/17 06:30
• Resolved: 23/Jun/16 03:48

Description

While testing found below issue. For some of the task counters, we do not have consistent values. This is not the case with every counter though.
Consider the case of counter "org.apache.hadoop.mapreduce.FileSystemCounter:FILE_BYTES_WRITTEN".

I found that its value for a flow I ran, was 936018 bytes. For the 3 apps associated with this flow run, the values were 312006 bytes each (which equals to value for a flow run i.e. 3 * 312006 = 936018). Drilling further down I found though that for one of the apps, the 4 tasks(2 mappers and 2 reducers) had values as 155918 bytes each for the 2 reducers and 156003 bytes each for the 2 mappers.

This means the value reported for the app should be (2 * 156003 + 2* 155918) or 623842 bytes but it is only 312006 bytes which indicates that only counter value of mappers is being picked up.