[HDDS-8507] Ozone Datanode fails with Metrics source path already exists

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 1.3.0
• Component/s: Ozone Datanode
• Assignee: Unassigned
• Reporter: Pratyush Bhatt
• Created: 28/Apr/23 11:30
• Updated: 12/Jul/23 07:11
• Resolved: 12/Jul/23 07:11

Description

Ozone Datanode fails with the below error:

Exception in HddsDatanodeService.
org.apache.hadoop.metrics2.MetricsException: Metrics source VolumeInfoMetrics-/grid/2/hadoop-ozone/datanode/data already exists!
	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newSourceName(DefaultMetricsSystem.java:152)
	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.sourceName(DefaultMetricsSystem.java:125)
	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.register(MetricsSystemImpl.java:229)
	at org.apache.hadoop.ozone.container.common.volume.VolumeInfoMetrics.init(VolumeInfoMetrics.java:50)
	at org.apache.hadoop.ozone.container.common.volume.VolumeInfoMetrics.<init>(VolumeInfoMetrics.java:45)
	at org.apache.hadoop.ozone.container.common.volume.HddsVolume.<init>(HddsVolume.java:140)
	at org.apache.hadoop.ozone.container.common.volume.HddsVolume.<init>(HddsVolume.java:73)
	at org.apache.hadoop.ozone.container.common.volume.HddsVolume$Builder.build(HddsVolume.java:115)
	at org.apache.hadoop.ozone.container.common.volume.HddsVolumeFactory.createFailedVolume(HddsVolumeFactory.java:60)
	at org.apache.hadoop.ozone.container.common.volume.MutableVolumeSet.initializeVolumeSet(MutableVolumeSet.java:187)
	at org.apache.hadoop.ozone.container.common.volume.MutableVolumeSet.<init>(MutableVolumeSet.java:135)
	at org.apache.hadoop.ozone.container.common.volume.MutableVolumeSet.<init>(MutableVolumeSet.java:99)
	at org.apache.hadoop.ozone.container.ozoneimpl.OzoneContainer.<init>(OzoneContainer.java:144)
	at org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.<init>(DatanodeStateMachine.java:173)
	at org.apache.hadoop.ozone.HddsDatanodeService.start(HddsDatanodeService.java:294)
	at org.apache.hadoop.ozone.HddsDatanodeService.start(HddsDatanodeService.java:222)
	at org.apache.hadoop.ozone.HddsDatanodeService.call(HddsDatanodeService.java:190)
	at org.apache.hadoop.ozone.HddsDatanodeService.call(HddsDatanodeService.java:84)
	at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
	at picocli.CommandLine.access$1300(CommandLine.java:145)
	at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
	at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
	at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
	at picocli.CommandLine.execute(CommandLine.java:2078)
	at org.apache.hadoop.hdds.cli.GenericCli.execute(GenericCli.java:100)
	at org.apache.hadoop.hdds.cli.GenericCli.run(GenericCli.java:91)
	at org.apache.hadoop.ozone.HddsDatanodeService.main(HddsDatanodeService.java:172)

Issue Links

• is duplicated by: HDDS-8914 Datanode may fail to start due to duplicate VolumeInfoMetrics