[AMBARI-25635] Clear Cluster and METRIC_AGGREGATORS MBeans upon shutdown

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.7.4, 2.7.5
• Fix Version/s: 2.8.0, 2.7.6
• Component/s: ambari-metrics
• Labels: HA helix metric-collector pull-request-available
• Assignee: Tamas Payer
• Reporter: Tamas Payer
• Created: 16/Mar/21 11:38
• Updated: 17/Nov/22 09:28
• Resolved: 28/Apr/21 06:28

Description

Following warnings appear in metrics-collector.log upon startup.

javax.management.InstanceAlreadyExistsException: ClusterStatus: cluster=ambari-metrics-cluster,instanceName=ctr-e153-1613480641811-87570-01-000057.hwx.site_12001
 at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
 at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
 at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
 at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
 at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
 at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
 at org.apache.helix.monitoring.mbeans.ClusterStatusMonitor.register(ClusterStatusMonitor.java:172)
 at org.apache.helix.monitoring.mbeans.ClusterStatusMonitor.registerInstances(ClusterStatusMonitor.java:498)
 at org.apache.helix.monitoring.mbeans.ClusterStatusMonitor.setClusterInstanceStatus(ClusterStatusMonitor.java:220)
 at org.apache.helix.controller.stages.ReadClusterDataStage.process(ReadClusterDataStage.java:91)
 at org.apache.helix.controller.pipeline.Pipeline.handle(Pipeline.java:48)
 at org.apache.helix.controller.GenericHelixController.handleEvent(GenericHelixController.java:295)
 at org.apache.helix.controller.GenericHelixController$ClusterEventProcessor.run(GenericHelixController.java:595)

and

2021-03-09 23:15:53,450 INFO org.apache.helix.monitoring.mbeans.ClusterStatusMonitor: Register MBean: ClusterStatus: cluster=ambari-metrics-cluster,instanceName=ctr-e153-1613480641811-87570-01-000057.hwx.site_12001,resourceName=METRIC_AGGREGATORS
2021-03-09 23:15:53,450 INFO org.apache.helix.monitoring.mbeans.ClusterStatusMonitor: Register MBean: ClusterStatus: cluster=ambari-metrics-cluster,instanceName=ctr-e153-1613480641811-87570-01-000057.hwx.site_12001,resourceName=METRIC_AGGREGATORS
2021-03-09 23:15:53,450 WARN org.apache.helix.monitoring.mbeans.ClusterStatusMonitor: Could not register MBean: ClusterStatus: cluster=ambari-metrics-cluster,instanceName=ctr-e153-1613480641811-87570-01-000057.hwx.site_12001,resourceName=METRIC_AGGREGATORS
javax.management.InstanceAlreadyExistsException: ClusterStatus: cluster=ambari-metrics-cluster,instanceName=ctr-e153-1613480641811-87570-01-000057.hwx.site_12001,resourceName=METRIC_AGGREGATORS
        at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
        at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
        at org.apache.helix.monitoring.mbeans.ClusterStatusMonitor.register(ClusterStatusMonitor.java:172)
        at org.apache.helix.monitoring.mbeans.ClusterStatusMonitor.registerPerInstanceResources(ClusterStatusMonitor.java:537)
        at org.apache.helix.monitoring.mbeans.ClusterStatusMonitor.setPerInstanceResourceStatus(ClusterStatusMonitor.java:307)
        at org.apache.helix.controller.stages.BestPossibleStateCalcStage.process(BestPossibleStateCalcStage.java:74)
        at org.apache.helix.controller.pipeline.Pipeline.handle(Pipeline.java:48)
        at org.apache.helix.controller.GenericHelixController.handleEvent(GenericHelixController.java:295)
        at org.apache.helix.controller.GenericHelixController$ClusterEventProcessor.run(GenericHelixController.java:595)

Issue Links

• links to: GitHub PR#3304
• links to: GitHub Pull Request #68