[HDDS-8734] OM DB cache metrics creation should be idempotent

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.4.0
• Component/s: Ozone Manager
• Labels: pull-request-available
• Assignee: Devesh Kumar Singh
• Reporter: Devesh Kumar Singh
• Created: 31/May/23 08:50
• Updated: 27/Sep/23 23:35
• Resolved: 01/Jun/23 09:32

Description

When a cache metrics being registered for any OM DB table, it should check if cache metrics already registered or not. If it is already registered, then we should unregister first and allow registration.

Duplicate metrics prevent Recon full snapshot update:

OzoneManagerServiceProviderImpl: Unable to update Recon's metadata with new OM DB.
org.apache.hadoop.metrics2.MetricsException: Metrics source userTableCache already exists!
        at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newSourceName(DefaultMetricsSystem.java:152)
        at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.sourceName(DefaultMetricsSystem.java:125)
        at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.register(MetricsSystemImpl.java:229)
        at org.apache.hadoop.hdds.utils.TableCacheMetrics.create(TableCacheMetrics.java:67)
        at org.apache.hadoop.hdds.utils.db.TypedTable.createCacheMetrics(TypedTable.java:332)
        at org.apache.hadoop.ozone.om.OmMetadataManagerImpl.checkTableStatus(OmMetadataManagerImpl.java:429)
        at org.apache.hadoop.ozone.om.OmMetadataManagerImpl.initializeOmTables(OmMetadataManagerImpl.java:559)
        at org.apache.hadoop.ozone.recon.recovery.ReconOmMetadataManagerImpl.initializeNewRdbStore(ReconOmMetadataManagerImpl.java:103)
        at org.apache.hadoop.ozone.recon.recovery.ReconOmMetadataManagerImpl.updateOmDB(ReconOmMetadataManagerImpl.java:120)
        at org.apache.hadoop.ozone.recon.spi.impl.OzoneManagerServiceProviderImpl.updateReconOmDBWithNewSnapshot(OzoneManagerServiceProviderImpl.java:384)
        at org.apache.hadoop.ozone.recon.spi.impl.OzoneManagerServiceProviderImpl.syncDataFromOM(OzoneManagerServiceProviderImpl.java:527)
        at org.apache.hadoop.ozone.recon.spi.impl.OzoneManagerServiceProviderImpl.lambda$startSyncDataFromOM$0(OzoneManagerServiceProviderImpl.java:258)

Issue Links

• links to: GitHub Pull Request #4805