[FLINK-25414] Provide metrics to measure how long task has been blocked

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.14.2
• Fix Version/s: 1.15.0
• Component/s: Runtime / Metrics, Runtime / Task
• Labels: pull-request-available
• Assignee: Piotr Nowojski
• Reporter: Piotr Nowojski
• Created: 22/Dec/21 08:59
• Updated: 15/Feb/22 10:18
• Resolved: 02/Jan/22 11:02

Description

https://lists.apache.org/thread/f6ffk9912fg5j1rfkxbzrh0qmp4w6qry

Currently back pressured/busy metrics tell the user whether task is blocked/busy and how much % of the time it is blocked/busy. But they do not tell how for how long single block event is lasting. It can be 1ms or 1h and back pressure/busy would be still reporting 100%.

In order to improve this, we could provide two new metrics:

1. maxSoftBackPressureTime
2. maxHardBackPressureTime

The max would be reset to 0 periodically or on every access to the metric (via metric reporter). Soft back pressure would be if task is back pressured in a non blocking fashion (StreamTask detected in availability of the output). Hard back pressure would measure the time task is actually blocked.

In order to calculate those metrics I'm proposing to split the already existing backPressuredTimeMsPerSecond into soft and hard versions as well.

Unfortunately I don't know how to efficiently provide similar metric for busy time, without impacting max throughput.

Issue Links

• links to: GitHub Pull Request #18181