Description
It seems that the patch for KAFKA-2015 caused a few changes in the behavior of the console consumer. I picked this up because it caused the new mirror maker sanity system test to hang. We need a separate fix for ducktape to address the lack of a timeout where it got stuck, but I'd also like to get this fixed ASAP since it affects pretty much all system test efforts since they commonly use console consumer to validate data produced to Kafka.
I've tracked down a couple of changes so far:
1. The --consumer.config option handling was changed entirely. I think the new approach was trying to parse it as key=value parameters, but it's supposed to be a properties file containing key=value pairs.
2. A few different exceptions during message processing are not handled the same way. The skipMessageOnErrorOpt is not longer being used at all (it's parsed, but that option is never checked anymore). Also, exceptions during iteration are not caught. After fixing the consumer.config issue, which was keeping the consumer.timeout.ms setting from making it into the consumer config, this also caused the process to hang. It killed the main thread, but there must be another non-daemon thread still running (presumably the consumer threads?)
3. The "consumed X messages" message changed from stderr to stdout.
Attachments
Issue Links
- is related to
-
KAFKA-2466 ConsoleConsumer throws ConcurrentModificationException on termination
-
- Resolved
-