[KAFKA-6717] TopicPartition Assined twice to a consumer group for 2 consumer instances

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 0.11.0.1
• Assignee: Unassigned
• Reporter: Yuancheng PENG
• Created: 27/Mar/18 09:28
• Updated: 17/Jul/18 19:07
• Resolved: 17/Jul/18 19:07

Description

I'm using StickyAssignor for consuming more than 100 topics with certain pattern.

There are 10 consumers with the same group id.

I expected that topic-partition to be assigned to only one consumer instance. However some topic partitions are assigned twice in 2 different difference instance, hence the consumer group process duplicate messages.

props.put(ConsumerConfig.PARTITION_ASSIGNMENT_STRATEGY_CONFIG, Collections.singletonList(StickyAssignor.class));

KafkaConsumer<Record, Record> c = new KafkaConsumer<>(props);
c.subscribe(Pattern.compile(TOPIC_PATTERN), new NoOpConsumerRebalanceListener());

 

 

Issue Links

• duplicates: KAFKA-7026 Sticky assignor could assign a partition to multiple consumers (KIP-341)