[FLINK-10243] Add option to reduce latency metrics granularity

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.7.0
• Fix Version/s: 1.5.5, 1.6.2, 1.7.0
• Component/s: Runtime / Configuration, Runtime / Metrics
• Labels: pull-request-available
• Release Note: The default granularity for latency metrics was modified. To restore the previous behavior users have to explicitly set the granularity to SUBTASK.
• Assignee: Chesnay Schepler
• Reporter: Chesnay Schepler
• Created: 29/Aug/18 07:20
• Updated: 02/Oct/19 17:49
• Resolved: 10/Oct/18 10:06

Description

The latency is currently tracked separately from each operator subtask to each source subtask. The total number of latency metrics in the cluster is thus (# of sources) * (# of operators) * parallelism², i.e. quadratic scaling.

If we'd ignore the source subtask the scaling would be a lot more manageable.

Issue Links

• links to: GitHub Pull Request #6658