[HADOOP-9128] MetricsDynamicMBeanBase can cause high cpu load

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.20.2
• Fix Version/s: 1.0.0, 0.23.0
• Component/s: metrics

Environment

cdh3u4

• Assignee: Unassigned
• Reporter: Nate Putnam
• Created: 11/Dec/12 00:51
• Updated: 16/Mar/15 18:01
• Resolved: 20/Dec/12 06:18

Description

I noticed high load on some of our Hadoop services. On closer inspection we found several threads that were consuming cpu (80-90% of a core) doing :

java.lang.Thread.State: RUNNABLE
at java.util.HashMap.get(HashMap.java:303)
at org.apache.hadoop.metrics.util.MetricsDynamicMBeanBase.getAttribute(MetricsDynamicMBeanBase.java:135)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:666)

This is a known issue in java when using a non thread safe hash map from multiple threads