[FLINK-12579] Influx Metric exporter throws a lot of negative infinity errors

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Component/s: Runtime / Metrics
• Assignee: Unassigned
• Reporter: Theo Diefenthal
• Created: 21/May/19 19:13
• Updated: 22/May/19 08:17
• Resolved: 22/May/19 08:17

Description

When using the InfluxDB metrics, the logs are polluted by lots of warnings due to negative infinity values which influxDB can't handle:

2019-05-21 18:24:40,410 WARN org.apache.flink.runtime.metrics.MetricRegistryImpl - Error while reporting metrics
org.apache.flink.metrics.influxdb.shaded.org.influxdb.InfluxDBException$UnableToParseException: partial write: unable to parse 'taskmanager_job_task_operator_sync-time-max,host=..,job_id=d7306cf5af0cf386a9259845d2a79f7c,job_name=..,operator_id=0be27db0efe2375e2766b48026cbee26,operator_name=Source:\ ...kafka_source...,subtask_index=0,task_attempt_id=bbff973b4e71d377745f0f2e3bf884ef,task_attempt_num=0,task_id=0be27db0efe2375e2766b48026cbee26,task_name=Source:\ ..._kafka_source_...,tm_id=container_e101_1557348638026_49850_01_000002 value=∞ 1558463080133000000': invalid number
unable to parse 'taskmanager_job_task_operator_KafkaConsumer_join-time-max,host=...,job_id=d7306cf5af0cf386a9259845d2a79f7c,job_name=...,operator_id=0be27db0efe2375e2766b48026cbee26,operator_name=Source:\ ...kafka_source...,subtask_index=0,task_attempt_id=bbff973b4e71d377745f0f2e3bf884ef,task_attempt_num=0,task_id=0be27db0efe2375e2766b48026cbee26,task_name=Source:\ ...kafka_source...,tm_id=container_e101_1557348638026_49850_01_000002 value=-∞ 1558463080133000000': invalid number
 ....
  unable to parse 'taskmanager_job_task_operator_KafkaConsumer_heartbeat-response-time-max,host=...,job_id=d7306cf5af0cf386a9259845d2a79f7c,job_name=...,operator_id=0be27db0efe2375e2766b48026cbee26,operator_name=Source:\ ..kafka_source..,subtask_index=0,task_attempt_id=bbff973b4e71d377745f0f2e3bf884ef,task_attempt_num=0,task_id=0be27db0efe2375e2766b48026cbee26,task_name=Source:\ ..kafka_source...,tm_id=container_e101_1557348638026_49850_01_000002 value=-∞ 1558463080133000000': invalid number dropped=0
 at org.apache.flink.metrics.influxdb.shaded.org.influxdb.InfluxDBException.buildExceptionFromErrorMessage(InfluxDBException.java:147)
 at org.apache.flink.metrics.influxdb.shaded.org.influxdb.InfluxDBException.buildExceptionForErrorState(InfluxDBException.java:173)
 at org.apache.flink.metrics.influxdb.shaded.org.influxdb.impl.InfluxDBImpl.execute(InfluxDBImpl.java:796)
 at org.apache.flink.metrics.influxdb.shaded.org.influxdb.impl.InfluxDBImpl.write(InfluxDBImpl.java:455)
 at org.apache.flink.metrics.influxdb.InfluxdbReporter.report(InfluxdbReporter.java:97)
 at org.apache.flink.runtime.metrics.MetricRegistryImpl$ReporterTask.run(MetricRegistryImpl.java:430)
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
 at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
 at java.lang.Thread.run(Thread.java:748)

Issue Links

• duplicates: FLINK-12147 InfluxDB throws errors if metric value is NaN/infinity