[IGNITE-22403] Algorithm improvement for kafka topic partition distribution

• Type: Improvement
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Component/s: extensions
• Labels: IEP-59 ise
• Assignee: Maksim Davydov
• Reporter: Maksim Davydov
• Created: 04/Jun/24 08:52
• Updated: 06/Jun/24 15:16

Description

Distribution for partitions from kafka topic over requested threads in AbstractKafkaToIgniteCdcStreamer (CDC Kafka to Ignite) is not uniform.

There might be cases with unequal load on threads, that reads data from kafka, which might result in overhead/bottleneck and slow workflow in general for the whole CDC with Kafka process.

Issue Links

• links to: GitHub Pull Request #272