[IGNITE-22403] CDC: Improve Kafka partition ranges assignment for appliers

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Component/s: extensions
• Labels: IEP-59 ise
• Release Note: Fixed partition ranges assignment for appliers of KafkaToIgnite streamers
• Assignee: Maksim Davydov
• Reporter: Maksim Davydov
• Created: 04/Jun/24 08:52
• Updated: 10/Jul/24 14:58
• Resolved: 10/Jul/24 14:58

Description

Distribution for partitions from kafka topic over requested threads in AbstractKafkaToIgniteCdcStreamer (CDC Kafka to Ignite) is not uniform.

There might be cases with unequal load on threads, that reads data from kafka, which might result in overhead/bottleneck and slow workflow in general for the whole CDC with Kafka process.

Issue Links

• links to: GitHub Pull Request #272