[KAFKA-13408] Add a new metric to track invalid task provided offset

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: connect
• Assignee: Unassigned
• Reporter: Nicolas Guyomar
• Created: 27/Oct/21 10:18
• Updated: 27/Oct/21 10:18

Description

Hi team,

Whenever a task provide invalid offset back to the framework, we log a warn message but we do not track yet a dedicated metric for those

In some situations where the Sink connector task implements the precommit method and has a bug in the way it tracks assigned topic/partition this could lead to a task not reporting offset at all thus no offset is being tracked on the consumer group in Kafka and can go undetected if we rely on the existing JMX beans for offset commit

This improvement Jira is to add a dedicated JMX Metric to be able to alert if such invalid offset are provided too often or for too long

 

Thank you