[HADOOP-15008] Metrics sinks may emit too frequently if multiple sink periods are configured

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.2.0, 3.0.0-beta1
• Fix Version/s: 3.1.0
• Component/s: metrics
• Hadoop Flags: Incompatible change, Reviewed
• Release Note: Previously if multiple metrics sinks were configured with different periods, they may emit more frequently than configured, at a period as low as the GCD of the configured periods. This change makes all metrics sinks emit at their configured period.
• Assignee: Erik Krogen
• Reporter: Erik Krogen
• Created: 01/Nov/17 21:08
• Updated: 13/Nov/17 23:39
• Resolved: 13/Nov/17 17:49

Description

If there are multiple metrics sink periods configured, depending on what those periods are, some sinks may emit too frequently. For example with the following:

hadoop-metrics2.properties

namenode.sink.file10.class=org.apache.hadoop.metrics2.sink.FileSink
namenode.sink.file5.class=org.apache.hadoop.metrics2.sink.FileSink
namenode.sink.file10.filename=namenode-metrics_per10.out
namenode.sink.file5.filename=namenode-metrics_per5.out
namenode.sink.file10.period=10
namenode.sink.file5.period=5

I get the following:

± for f in namenode-metrics_per*.out; do echo "$f" && grep "metricssystem.MetricsSystem" $f | awk '{last=curr; curr=$1} END { print curr-last }'; done
namenode-metrics_per10.out
5000
namenode-metrics_per5.out
5000

i.e., for both metrics files, each record is 5000 ms apart, even though one of the sinks has been configured to emit at 10s intervals