[FLINK-31599] Update Kafka dependency in flink-connector-kafka to 3.4.0

• Type: Technical Debt
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: kafka-3.1.0
• Component/s: Connectors / Kafka
• Labels: pull-request-available
• Assignee: Alex Sorokoumov
• Reporter: Alex Sorokoumov
• Created: 23/Mar/23 18:12
• Updated: 12/Apr/23 17:02
• Resolved: 24/Mar/23 20:33

Description

There is a number of reasons to upgrade to the latest version.
 
First, the Kafka connector uses reflection, so internal changes in Kafka clients' implementation might require changes in the connector. With more frequent upgrades, the amount of work per upgrade is smaller.
 
Second, there were a number of relevant bug fixes since 3.2.3:

• KAFKA-14303 - Producer.send without record key and batch.size=0 goes into infinite loop
• KAFKA-14379 - consumer should refresh preferred read replica on update metadata
• KAFKA-14422 - Consumer rebalance stuck after new static member joins a group with members not supporting static members
• KAFKA-14417 - Producer doesn't handle REQUEST_TIMED_OUT for InitProducerIdRequest, treats as fatal error
• KAFKA-14532 - Correctly handle failed fetch when partitions unassigned
  * 

Issue Links

• links to: GitHub Pull Request #11