[KAFKA-13017] Excessive logging on sink task deserialization errors

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.7.0, 2.8.0, 3.0.0
• Fix Version/s: 3.1.0
• Component/s: connect
• Assignee: Chris Egerton
• Reporter: Chris Egerton
• Created: 30/Jun/21 15:20
• Updated: 16/Jul/21 14:57
• Resolved: 16/Jul/21 14:40

Description

Even with errors.log.enable set to false, deserialization failures are still logged at ERROR level by the org.apache.kafka.connect.runtime.WorkerSinkTask namespace. This becomes problematic in pipelines with errors.tolerance set to all, and can generate excessive logging of stack traces when deserialization errors are encountered for most if not all of the records being consumed by a sink task.

The logging added to the WorkerSinkTask class in KAFKA-9018 should be removed and, if necessary, any valuable information from it not already present in the log messages generated by Connect with errors.log.enable and errors.log.include.messages set to true should be added in that place instead.

Issue Links

• is caused by: KAFKA-9018 Kafka Connect - throw clearer exceptions on serialisation errors
• links to: GitHub Pull Request #10945