Blocker Description
We discovered that when KafkaConsumer subscribes to multiple topics with max.poll.record configured. The max.poll.record is not properly respected for all poll() invocation.
I was able to reproduce it with the AK example, here is how I ran my tests:
https://github.com/apache/kafka/pull/14772
1. start zookeeper and kafka server (or kraft mode should be fine too)
2. Run: examples/bin/java-producer-consumer-demo.sh 10000000
3. Polled records > 400 will be printed to stdout
Here is what the program does:
The produce produces a large number of records to multiple topics. We configure the consumer using a max.poll.record = 400, and subscribed to multiple topics. The consumer poll, and the returned records can sometimes be larger than 400.
This is an issue in AK 3.6 but 3.5 was fine.
Attachments
Issue Links
- links to