Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Incomplete
-
2.2.3, 2.3.3, 2.4.2
-
None
Description
Currently Spark allows registering multiple Metric Sources with the same source name like the following
val acc1 = sc.longAccumulator LongAccumulatorSource.register(sc, {"acc1" -> acc1}) val acc2 = sc.longAccumulator LongAccumulatorSource.register(sc, {"acc2" -> acc2})
In that case there are two metric sources registered and both of these sources have the same name - AccumulatorSource
If you try to unregister the source with no accumulators and metrics registered like the following
SparkEnv.get.metricsSystem.removeSource(new LongAccumulatorSource)
... then all the metrics for all the sources with the same name will be unregistered because of the following snippet which removes all matching records which start with the corresponding prefix which includes the source name, but does not include metric name to be removed.
def removeSource(source: Source) { sources -= source val regName = buildRegistryName(source) registry.removeMatching((name: String, _: Metric) => name.startsWith(regName)) }