[FLINK-33681] Display source/sink numRecordsIn/Out & numBytesIn/Out on UI

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.18.0, 1.17.2, 1.20.0, 1.19.1
• Fix Version/s: 2.0-preview
• Component/s: Runtime / Metrics
• Labels: pull-request-available
• Release Note: The Source and Sink IO metrics are not reported to the tasks. This means the Web UI will show the IO metrics of the source and sink vertices, instead of 0.
• Assignee: Zhanghao Chen
• Reporter: Zhanghao Chen
• Created: 29/Nov/23 05:30
• Updated: 02/Sep/24 09:30
• Resolved: 21/Aug/24 22:36

Description

Currently, the numRecordsIn & numBytesIn metrics for sources and the numRecordsOut & numBytesOut metrics for sinks are always 0 on the Flink web dashboard.

FLINK-11576 brings us these metrics on the opeartor level, but it does not integrate them on the task level. On the other hand, the summay metrics on the job overview page is based on the task level I/O metrics. As a result, even though new connectors supporting FLIP-33 metrics will report operator-level I/O metrics, we still cannot see the metrics on dashboard.

This ticket serves as an umbrella issue to integrate standard source/sink I/O metrics with the corresponding task I/O metrics. 

Issue Links

• is related to: FLINK-11576 FLIP-33: Standardize connector metrics
• is related to: FLINK-7286 Flink Dashboard fails to display bytes/records received by sources / emitted by sinks
• links to: GitHub Pull Request #23998
• mentioned in: Page Loading...