[YARN-6212] NodeManager metrics returning wrong negative values

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 2.7.3
• Component/s: metrics
• Assignee: Unassigned
• Reporter: Abhishek Shivanna
• Created: 22/Feb/17 02:30
• Updated: 07/Jan/20 09:17
• Resolved: 22/Feb/17 18:01

Description

It looks like the metrics returned by the NodeManager have negative values for metrics that never should be negative. Here is an output form NM endpoint

/jmx?qry=Hadoop:service=NodeManager,name=NodeManagerMetrics

{
  "beans" : [ {
    "name" : "Hadoop:service=NodeManager,name=NodeManagerMetrics",
    "modelerType" : "NodeManagerMetrics",
    "tag.Context" : "yarn",
    "tag.Hostname" : "<HOST>",
    "ContainersLaunched" : 707,
    "ContainersCompleted" : 9,
    "ContainersFailed" : 124,
    "ContainersKilled" : 579,
    "ContainersIniting" : 0,
    "ContainersRunning" : 19,
    "AllocatedGB" : -26,
    "AllocatedContainers" : -5,
    "AvailableGB" : 252,
    "AllocatedVCores" : -5,
    "AvailableVCores" : 101,
    "ContainerLaunchDurationNumOps" : 718,
    "ContainerLaunchDurationAvgTime" : 18.0
  } ]
}

Is there any circumstance under which the value for AllocatedGB, AllocatedContainers and AllocatedVCores go below 0?

Issue Links

• duplicates: YARN-3933 FairScheduler: Multiple calls to completedContainer are not safe
• relates to: YARN-6966 NodeManager metrics may return wrong negative values when NM restart