Description
I noticed high load on some of our Hadoop services. On closer inspection we found several threads that were consuming cpu (80-90% of a core) doing :
java.lang.Thread.State: RUNNABLE
at java.util.HashMap.get(HashMap.java:303)
at org.apache.hadoop.metrics.util.MetricsDynamicMBeanBase.getAttribute(MetricsDynamicMBeanBase.java:135)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:666)
This is a known issue in java when using a non thread safe hash map from multiple threads