Details
Major Description
When creating a simple stream topology to just populate a GlobalKTable, I noticed from logging that the stream consumer was doing group coordination requests (JoinGroup, SyncGroup, ...) to the server, which it had no reason to do since the global consumer thread populating the table fetches from all partitions and thus doesn't use the group requests. So this adds needless overhead on the client, network, and server.
I tracked this down to the stream thread consumer, which is created regardless of whether it's needed based solely on NUM_STREAM_THREADS_CONFIG which defaults to 1 I guess.
I found that setting NUM_STREAM_THREADS_CONFIG to 0 will prevent this from happening, but it'd be a worthwhile improvement to be able to override this setting in cases of topologies like this that don't have any need for stream threads. Hence this ticket.
I originally asked about this on the users mailing list where Bruno suggested I file it as an improvement request.
Here's the Scala code that I'm using that exhibits this:
val builder: StreamsBuilder = new StreamsBuilder() val gTable = builder.globalTable[K, V](...) val stream = new KafkaStreams(builder.build(), props) stream.start()
Not shown is the state store that I'm populating/using.