[FLINK-11887] Latency metrics drift apart

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.6.3
• Fix Version/s: 1.7.3, 1.8.0
• Component/s: Runtime / Metrics
• Labels: pull-request-available
• Assignee: Suxing Lee
• Reporter: Suxing Lee
• Created: 12/Mar/19 13:05
• Updated: 05/Apr/19 12:00
• Resolved: 22/Mar/19 10:40

Description

The operator's latency time is increased by approximately 2.7 minutes per day (see the attached).
We compute the latency by System.currentTimeMillis - marker.getMarkedTime.
There is no guarantee that System.currentTimeMillis and System.nanoTime don't drift apart.
If a GC pause or linux preemptive scheduling happenes, this should affect latency metrics.
Latency metrics drift away from their initial values with time(verify this result via the JVM Heap Dump).

Issue Links

• links to: GitHub Pull Request #7966