[KAFKA-2868] retry.backoff.ms ignored by Producer

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 0.8.2.1
• Component/s: producer
• Environment: Host : Ubuntu 14.04 LTS x86_64 (3.16.0-53-generic); Container LXC : Centos 6
• Assignee: Jun Rao
• Reporter: Nicolas Bordier
• Created: 20/Nov/15 14:24
• Updated: 12/May/16 10:36
• Resolved: 12/May/16 10:36

Description

In our test, the Producer config parameter is defined as :
retry.backoff.ms = 10000

During kafka partition reassignement, the producer log NOT_LEADER_FOR_PARTITION errors not every 10s (as defined in retry.backoff.ms) but very quickly (every 100ms maximum) :

[2015-11-17 11:53:30,002] WARN Got error produce response with correlation id 23080 on topic-partition raw-9, retrying (99 attempts left). Error: NOT_LEADER_FOR_PARTITION (org.apache.kafka.clients.producer.internals.Sender:257)
[2015-11-17 11:53:30,037] WARN Got error produce response with correlation id 23081 on topic-partition raw-9, retrying (98 attempts left). Error: NOT_LEADER_FOR_PARTITION (org.apache.kafka.clients.producer.internals.Sender:257)
[2015-11-17 11:53:30,260] WARN Got error produce response with correlation id 23083 on topic-partition raw-9, retrying (97 attempts left). Error: NOT_LEADER_FOR_PARTITION (org.apache.kafka.clients.producer.internals.Sender:257)

In our test, the retry.backoff.ms parameter is ignored by Producer.

We need to configure this parameter to avoid errors (ie : "retrying (0 attempts left)" like during a long time partition reassignement).

Thanks,

Issue Links

• is duplicated by: KAFKA-2138 KafkaProducer does not honor the retry backoff time.