[FLINK-10715] E2e tests fail with ConcurrentModificationException in MetricRegistryImpl

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.7.0
• Fix Version/s: 1.5.6, 1.6.3, 1.7.0
• Component/s: Runtime / Metrics, Test Infrastructure
• Labels: pull-request-available
• Assignee: Igal Shilman
• Reporter: Dawid Wysakowicz
• Created: 29/Oct/18 12:35
• Updated: 22/Nov/18 09:21
• Resolved: 03/Nov/18 09:37

Description

Couple of e2e tests that rely on metrics fail with exception:

2018-10-29 11:40:32,781 WARN  org.apache.flink.runtime.metrics.MetricRegistryImpl           - Error while reporting metrics
java.util.ConcurrentModificationException
	at java.util.HashMap$HashIterator.nextNode(HashMap.java:1437)
	at java.util.HashMap$EntryIterator.next(HashMap.java:1471)
	at java.util.HashMap$EntryIterator.next(HashMap.java:1469)
	at org.apache.flink.metrics.slf4j.Slf4jReporter.report(Slf4jReporter.java:101)
	at org.apache.flink.runtime.metrics.MetricRegistryImpl$ReporterTask.run(MetricRegistryImpl.java:427)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Tests that failed:

• 'Resuming Externalized Checkpoint (file, sync, no parallelism change) end-to-end test
• 'State TTL Heap backend end-to-end test'

Issue Links

• is duplicated by: FLINK-10035 ConcurrentModificationException with flink-metrics-slf4j
• links to: GitHub Pull Request #6996
• links to: GitHub Pull Request #7008