The statsd reporter has a bug I believe when using negative numbers.
This is because when a metric is sent it is first converted to a string value. This means 100 becomes "100" and -100 becomes "-100". This value is then sent to statsd as a gauge value.
See this line for the conversion to string: https://github.com/apache/flink/blob/master/flink-metrics/flink-metrics-statsd/src/main/java/org/apache/flink/metrics/statsd/StatsDReporter.java#L130
And this line for sending it as a gauge:
This means a value of -100 will be sent like this:
The statsd protocol how ever states the following (https://github.com/statsd/statsd/blob/master/docs/metric_types.md#gauges):
So sending -100 multiple times means the gauge in statsd will be decremented multiple times, rather than set to -100.
I believe this isn't how flink expects it to work, is it?