[KAFKA-9971] Error Reporting in Sink Connectors

• Type: Improvement
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.6.0
• Fix Version/s: 2.6.0
• Component/s: connect
• Assignee: Aakash Shah
• Reporter: Aakash Shah
• Reviewer: Randall Hauch
• Created: 07/May/20 23:34
• Updated: 02/Aug/23 16:48
• Resolved: 28/May/20 13:47

Description

Currently, KIP-298 provides error handling in Kafka Connect that includes functionality such as retrying, logging, and sending errant records to a dead letter queue. However, the dead letter queue functionality from KIP-298 only supports error reporting within contexts of the transform operation, and key, value, and header converter operation. Within the context of the put(...) method in sink connectors, there is no support for dead letter queue/error reporting functionality. 

Issue Links

• is duplicated by: KAFKA-9314 Connect put() and poll() retries not conforming to KIP-298
• is related to: KAFKA-10111 SinkTaskContext.errantRecordReporter() added in KIP-610 should be a default method
• is related to: KAFKA-9314 Connect put() and poll() retries not conforming to KIP-298
• is related to: KAFKA-10115 Incorporate errors.tolerance with the Errant Record Reporter
• supercedes: KAFKA-8597 Give access to the Dead Letter Queue APIs to Kafka Connect Developers
• links to: GitHub Pull Request #8720
• mentioned in: Page Loading...
• mentioned in: Page Loading...