[YARN-3619] ContainerMetrics unregisters during getMetrics and leads to ConcurrentModificationException

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.7.0
• Fix Version/s: 2.8.0, 2.7.2, 3.0.0-alpha1
• Component/s: nodemanager
• Target Version/s: 2.7.2
• Hadoop Flags: Reviewed
• Assignee: Zhihai Xu
• Reporter: Jason Darrell Lowe
• Created: 11/May/15 20:18
• Updated: 06/Jan/17 01:32
• Resolved: 02/Oct/15 20:26

Description

ContainerMetrics is able to unregister itself during the getMetrics method, but that method can be called by MetricsSystemImpl.sampleMetrics which is trying to iterate the sources. This leads to a ConcurrentModificationException log like this:

2015-05-11 14:00:20,360 [Timer for 'NodeManager' metrics system] WARN impl.MetricsSystemImpl: java.util.ConcurrentModificationException

Issue Links

• contains: YARN-4563 ContainerMetrics deadlocks
• is broken by: YARN-2984 Metrics for container's actual memory usage