[HADOOP-13362] DefaultMetricsSystem leaks the source name when a source unregisters

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 2.7.2
• Fix Version/s: 2.7.4
• Component/s: metrics
• Target Version/s: 2.7.4
• Hadoop Flags: Reviewed
• Assignee: Junping Du
• Reporter: Jason Darrell Lowe
• Created: 08/Jul/16 20:48
• Updated: 29/Mar/17 22:15
• Resolved: 29/Mar/17 22:15

Description

Ran across a nodemanager that was spending most of its time in GC. Upon examination of the heap most of the memory was going to the map of names in org.apache.hadoop.metrics2.lib.UniqueNames. In this case the map had almost 2 million entries. Looking at a few of the map showed entries like "ContainerResource_container_e01_1459548490386_8560138_01_002020", "ContainerResource_container_e01_1459548490386_2378745_01_000410", etc.

Looks like the ContainerMetrics for each container will cause a unique name to be registered with UniqueNames and the name will never be unregistered.

Issue Links

• is duplicated by: YARN-5482 ContainerMetric Lead to memory leaks
• is related to: YARN-5296 NMs going OutOfMemory because ContainerMetrics leak in ContainerMonitorImpl
• relates to: YARN-5190 Registering/unregistering container metrics triggered by ContainerEvent and ContainersMonitorEvent are conflict which cause uncaught exception in ContainerMonitorImpl