[FLINK-36278] Fix Kafka connector logs getting too big

• Type: Technical Debt
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: kafka-3.3.0
• Fix Version/s: kafka-3.3.0
• Component/s: Connectors / Kafka
• Labels: pull-request-available
• Assignee: Arvid Heise
• Reporter: Arvid Heise
• Created: 13/Sep/24 13:02
• Updated: 23/Sep/24 12:19
• Resolved: 23/Sep/24 12:19

Description

Currently, on failures, it's not (easily) possible to investigate logs. The zip is over 40 mb and downloading fails.

It seems as if the container logs make up a huge chunk but should not be of interest in most cases. This ticket should be about reducing the logs size to normal levels (less than 1mb zipped). For comparison, a normal Flink run results in 3mb zipped logs with many more tests.

Issue Links

• links to: GitHub Pull Request #119