[CAMEL-13215] camel-kafka -consumerCount vs consumerStreams problem with pool size

• Type: Task
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.23.1, 3.11.2
• Fix Version/s: 3.12.0
• Component/s: camel-kafka
• Estimated Complexity: Unknown
• Assignee: Otavio Rodolfo Piske
• Reporter: Pablo
• Created: 18/Feb/19 13:23
• Updated: 21/Sep/21 12:11
• Resolved: 21/Sep/21 12:11

Description

Hi guys,

Currently we are configuring camel-kafka component, but we don't understand at all the consumer properties related to concurrency.

According to the configuration:

    consumersCount: The number of consumers that connect to kafka server (default 1)
    consumerStreams: Number of concurrent consumers on the consumer (default 10)

But... Having a look to the source code, the consumerStreams is the number of threads the executor is configured.

And a task is submitted to the executor for each kafka consumer, which is the consumerCount.

So... I don't see what you get if you configure different values in consumerCount and consumerStreams with the current implementation.

Basically I expected that if define consumerCount=1 and consumerStreams=10, which is the default. I would get 10 threads consuming from that Kafka consumer. But I end up having a pool of 10 threads with only a worker which is a single thread consumer.

 

Talking with Andrea Cosentino, seems that this maybe it's a problem in the documentation.

Issue Links

• relates to: CAMEL-16981 camel-kafka: improve the documentation about consumer streams vs consumers count
• links to: GitHub Pull Request #6129