Details
-
Improvement
-
Status: Resolved
-
Blocker
-
Resolution: Invalid
-
2.7.0
-
None
-
None
Description
In 2.7.0 release, producer config `retries` gets deprecated via KIP-572.
Connect is still using this config what needs to be fixed (cf https://github.com/apache/kafka/pull/8864/files#r439685920)
Btw: @hachikuji raise a concern about this issue, too: https://github.com/apache/kafka/pull/8864#pullrequestreview-443424531
> I just had one question about the proposal. Using retries=0 in the producer allows the user to have "at-most-once" delivery. This allows the application to implement its own retry logic for example. Do we still have a way to do this once this configuration is gone?
So maybe we need to do some follow up work in the `Producer` to make it work for Connect. But I would defer this to the follow up work.
My original though was, that setting `max.deliver.timeout.ms := request .timeout.ms` might prevent internal retries. But we need to verify this. It was also brought to my attention, that this might not work if the network disconnects – only `retries=0` would prevent to re-open the connection but a low timeout would not prevent retries.
In KIP-572, we proposed for Kafka Streams itself, to treat `task.timeout.ms = 0` as "no retries" – maybe we can do a similar thing for the producer?
There is also `max.block.ms` that we should consider. Unfortunately, I am not an expert on the `Producer`. But I would like to move forward with KIP-572 for now and are happy to help to resolve those questions.