[AMBARI-5607] Yarn Nodemanager Metrics only update every few minutes

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.5.1
• Fix Version/s: 1.6.1

Environment

sles11sp1

• Assignee: Dmytro Sen
• Reporter: Michael Harp
• Created: 29/Apr/14 02:42
• Updated: 27/Jun/14 00:06
• Resolved: 27/Jun/14 00:06

Description

Yarn Nodemanager Metrics take far too long between updates.

To demonstrate:

Run Terasort or anything that runs mapreduce:

hdfs dfs -mkdir -p benchmarks/terasort
hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar teragen -Dmapred.map.tasks=72 -Dmapred.reduce.tasks=36  1000000 benchmarks/terasort/input
hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar terasort -Dmapred.map.tasks=72 -Dmapred.reduce.tasks=36 benchmarks/terasort/input benchmarks/terasort/output
hdfs dfs -rm -R -skipTrash benchmarks/terasort

Then repeatedly probe the API at:

https://<server>:8081/api/v1/clusters/c1/services/YARN/components/NODEMANAGER?fields=host_components/metrics/yarn

It usually takes 2-3 minutes to see the metrics update, very repeatable.

Issue Links

• is related to: AMBARI-5974 Customize rrdcached configuration to improve io operations
• links to: Review Board