Details
-
Improvement
-
Status: In Progress
-
Trivial
-
Resolution: Unresolved
-
3.2.2
-
None
-
None
Description
Spark StreamingSource Metrics sourceName is inappropriate.The label now looks like `application_xxxxx_xxxx_driver_NetworkWordCount_StreamingMetrics_streaming_lastCompletedBatch_processingEndTime `, instead of `application_xxxxx_xxxx_driver_StreamingMetrics_streaming_lastCompletedBatch_processingEndTime`, the Spark app name is not need.
This makes it hard to use metrics for different Spark applications over time. And this makes the metrics sourceName standard inconsistent.
//代码占位符 private[streaming] class StreamingSource(ssc: StreamingContext) extends Source { override val metricRegistry = new MetricRegistry override val sourceName = "%s.StreamingMetrics".format(ssc.sparkContext.appName) .... }
And for example, other metrics sourceName don't have appName.
//代码占位符 private[spark] class LiveListenerBusMetrics(conf: SparkConf) extends Source with Logging { override val sourceName: String = "LiveListenerBus" override val metricRegistry: MetricRegistry = new MetricRegistry ... }