[KAFKA-8485] Kafka connect worker does not respond/function when kafka broker goes down.

• Type: Bug
• Status: Open
• Priority: Blocker
• Resolution: Unresolved
• Affects Version/s: 2.2.1
• Component/s: connect
• Labels: performance
• Flags: Important
• Assignee: Unassigned
• Reporter: kaushik srinivas
• Created: 05/Jun/19 06:00
• Updated: 07/Jun/19 07:51

Description

Below is the scenario

3 kafka brokers are up and running.

Kafka connect worker is installed and a hdfs sink connector is added.

Data streaming started, data being flushed out of kafka into hdfs.

Topic is created with 3 partitons, one leader on all the three brokers.

Now, 2 kafka brokers are restarted. Partition re balance happens.

Now we observe, kafka connect does not respond. REST API keeps timing out. 

Nothing useful is being logged at the connect logs as well.

Only way to get out of this situation currently is to restart the kafka connect worker and things gets normal.

 

The same scenario when tried without data being in progress, works fine. Meaning REST API does not get into timing out state. 

making this issue a blocker, because of the impact due to kafka broker restart.

 

Issue Links

• is related to: KAFKA-7941 Connect KafkaBasedLog work thread terminates when getting offsets fails because broker is unavailable