[FLINK-20724] Create a http handler for aggregating metrics from whole job

• Type: Improvement
• Status: Closed
• Priority: Not a Priority
• Resolution: Abandoned
• Affects Version/s: 1.13.0, 1.12.3
• Component/s: Runtime / Metrics
• Labels: auto-deprioritized-major stale-minor
• Assignee: Unassigned
• Reporter: Piotr Nowojski
• Created: 22/Dec/20 14:33
• Updated: 22/Nov/21 16:07
• Resolved: 22/Nov/21 16:07

Description

This is an optimisation idea.

Create a similar http handler to AggregatingSubtasksMetricsHandler, but one that would aggregate metrics per task, from all of the job vertices. The new handler would only take JobID as a parameter. So that Web UI can in one RPC obtain max(isBackPressureRatio) / max(isCausingBackPressureRatio) per each task in the job graph.

This is related to FLINK-14712, where we are invoking more REST calls to get the statistics per each task/node (in order to color the nodes based on the back pressure and busy times). With this new handler, WebUI could make a one REST call to get all the metrics it needs, instead of doing one REST call per every Task.

Issue Links

• is related to: FLINK-14712 Improve back-pressure reporting mechanism