[FLINK-31299] PendingRecords metric might not be available

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: kubernetes-operator-1.5.0
• Component/s: Autoscaler, Kubernetes Operator
• Labels: pull-request-available
• Release Note: Work around idle Kafka readers not emitting the pendingRecords metric
• Assignee: Maximilian Michels
• Reporter: Maximilian Michels
• Created: 02/Mar/23 12:57
• Updated: 06/Mar/23 16:34
• Resolved: 06/Mar/23 16:34

Description

The Kafka pendingRecords metric is only initialized on receiving the first record. For empty topics or checkpointed topics without any incoming data, the metric won't appear.

We need to handle this case in the autoscaler and allow downscaling.

Issue Links

• links to: GitHub Pull Request #542