[FLINK-34025] Show data skew score on Flink Dashboard

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Done
• Affects Version/s: 1.20.0
• Component/s: Runtime / Web Frontend
• Labels: dashboard pull-request-available
• Assignee: Emre Kartoglu
• Reporter: Emre Kartoglu
• Created: 08/Jan/24 14:38
• Updated: 05/Apr/24 14:38
• Resolved: 05/Apr/24 14:38

Description

Problem: Currently users have to click on every operator and check how much data each subtask is processing to see if there is data skew. This is particularly cumbersome and error-prone for jobs with big job graphs. Data skew is an important metric that should be more visible.

 

Proposed solution:

• Show a data skew score on each operator (see screenshot below). This would be an improvement, but would not be sufficient. As it would still not be easy to see the data skew score for jobs with very large job graphs (it'd require a lot of zooming in/out).
• Show data skew score for each operator under a new "Data Skew" tab next to the Exceptions tab. See screenshot below .

 

Issue Links

• links to: GitHub Pull Request #24598
• mentioned in: Page Loading...