[FLINK-27318] KafkaSink: when init transaction, it take too long to get a producerId with epoch=0

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 1.14.4
• Component/s: Connectors / Kafka
• Flags: Important
• Assignee: Unassigned
• Reporter: Zhengqi Zhang
• Created: 20/Apr/22 09:42
• Updated: 01/Jul/22 07:21
• Resolved: 01/Jul/22 07:21

Description

as we can see, the new KafkaSink aborts all transactions that have been created by a subtask in a previous run, only return when get a producerId was unused before(epoch=0). But this can take a long time, especially if the task has been started and cancelled many times before. In my tests, it even took 10 minutes. Is there a better way to solve this problem, or do what FlinkKafkaProducer did.