[FLINK-30464] Metrics are collected in stabilization phase and do not always span a full metric window

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: kubernetes-operator-1.4.0
• Fix Version/s: kubernetes-operator-1.4.0
• Component/s: Kubernetes Operator
• Labels: pull-request-available
• Assignee: Maximilian Michels
• Reporter: Maximilian Michels
• Created: 20/Dec/22 12:05
• Updated: 03/Jan/23 17:37
• Resolved: 03/Jan/23 17:37

Description

The current code collects metrics as soon as the job goes into RUNNING mode, regardless of whether we are still in the stabilization period. This means the first metric collection window after rescaling will include metrics from the stabilization period which is not desired. The stabilization period is supposed to allow the job to stabilize and shouldn't be used to make new scaling decisions.

Further, the collected metrics for the scaling decider initially do not span a full metric window. Only after the first metric window is complete, we are guaranteed to have a full windows available. This leads to short-sighted scaling decisions.

Issue Links

• is related to: FLINK-30463 Stabilization period starts before job goes into RUNNING
• relates to: FLINK-30510 Allow configuring minimum and maximum metrics window size
• links to: GitHub Pull Request #493