[HADOOP-15230] org.apache.hadoop.metrics2.GraphiteSink is not implemented correctly

• Type: Bug
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Component/s: metrics
• Labels: GraphiteSink, metrics pull-request-available

Release Note

Corrected problems with GraphiteSink not having proper format in point tags. Format v1.1 suggests point tags should be delimited with ';' rather than '.' and also appear at the end of the metric name, not in the middle. Also removed risk of point tag value having empty space (will be replaced by '_').

• Tags: metrics graphite
• Flags: Patch
• Assignee: Howard Yoo
• Reporter: Howard Yoo
• Created: 14/Feb/18 00:20
• Updated: 07/Feb/21 22:29

Description

org.apache.hadoop.metrics2.GraphiteSink's implementation has certain problems that would make it to generate metrics incorrectly.

The problem lies with line 77 ~ 84 of the GraphiteSink java:

for (MetricsTag tag : record.tags()) {
if (tag.value() != null) {
metricsPathPrefix.append(".");
metricsPathPrefix.append(tag.name());
metricsPathPrefix.append("=");
metricsPathPrefix.append(tag.value());
}
}

It produces point tags having name=value pair in the metrics. However, notice how the tags are added with '.' as its delimiters. Rather than using the '.' character, it should follow the following convention mentioned in the latest graphite doc of using ';' character.

http://graphite.readthedocs.io/en/latest/tags.html

Also, the value is not properly being escaped, meaning that if the value has a '.' character in it, it will easily confuse Graphite to accept it as a delimiter, rather than the value. A really good prime example is when the value is a hostname or ip address,

metrics.example.Hostname=this.is.a.hostname.and.this.is.Metrics 10.0

In this example, the since the value of the hostname contains '.', it is extremely hard for the receiving end to determine which part is hostname and which part is the rest of the metrics name. A good strategy is to convert any '.' character in the value to be converted to other characters, such as '_'.

However, the best way would be to follow the latest metrics convention of using ';'

metrics.example.and.this.is.Metrics;Hostname=this.is.a.hostname 10.0

Issue Links

• causes: HADOOP-10727 GraphiteSink metric names should not contain "."
• is related to: HADOOP-12538 Fix issues of metric path emmitted by GraphiteSink
• is related to: HADOOP-14300 GraphiteSink reports metrics containing undesirable whitespaces
• is related to: HADOOP-11237 GraphiteSink metric names should not contain "="
• links to: GitHub Pull Request #340
• links to: GitHub Pull Request #340