[FLINK-31557] Metric viewUpdater and reporter task in a SingleThreadScheduledExecutor lead to inaccurate PerSecond related metrics

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 1.18.0, 1.17.1
• Component/s: Runtime / Metrics
• Labels: pull-request-available
• Assignee: !huwh
• Reporter: LiuZeshan
• Created: 22/Mar/23 03:06
• Updated: 30/Mar/23 13:57
• Resolved: 30/Mar/23 13:56

Description

Currently, metric viewUpdater and reporterTask share the same SingleThreadScheduledExecutor, and customized reporters may have unpredictable logic, such as unreasonable network timeout settings, which can affect viewUpdater's calculation of PerSecond related metrics. For example, a real online problem we encountered, the network timeout of the reporter is set to 10 seconds, and the reporting interval is 15 seconds. When the server is unavailable, the thread is blocked for 10s, resulting in 66.7% (5/3x) higher PerSecond related metrics.

Is it possible to optimize here, such as whether it can be changed to a ScheduledThreadPool executor?

Issue Links

• links to: GitHub Pull Request #22239