Major Description
Kafka Streams supports `ProductionExceptionHandler` to drop records on error when writing into an output topic.
However, if the output topic does not exist, the corresponding error cannot be skipped over because the handler is not called.
The issue is, that the producer internally retires to fetch the output topic metadata until it times out, an a `TimeoutException` (which is a `RetriableException`) is returned via the registered `Callback`. However, for `RetriableException` there is different code path and the `ProductionExceptionHandler` is not called.
In general, Kafka Streams correctly tries to handle as many errors a possible internally, and a `RetriableError` falls into this category (and thus there is no need to call the handler). However, for this particular case, just retrying does not solve the issue – it's unclear if throwing a retryable `TimeoutException` is actually the right thing to do for the Producer? Also not sure what the right way to address this ticket would be (currently, we cannot really detect this case, except if we would do some nasty error message String comparison what sounds hacky...)
Attachments
Issue Links
- is blocked by
-
KAFKA-16965 Add a "root cause" exception as a nested exception to TimeoutException for Producer
-
- Resolved
-
- links to
- mentioned in
-
Page Loading...