[BEAM-12880] KafkaIO Connector-updating/overwriting consumer groupid with random prefix

• Type: Bug
• Status: Open
• Priority: P3
• Resolution: Unresolved
• Affects Version/s: 2.27.0
• Component/s: io-java-kafka

Language

java

• Assignee: Unassigned
• Reporter: Logeshwaran
• Created: 14/Sep/21 12:07
• Updated: 04/Jun/22 21:49

Description

Apache beam version: 2.27

Connector: KafkaIO

cloud service : GCP Dataflow

language : JAVA11

we are trying to read the avro messages from confluent kafka topic using dataflow service as a consumer. (Using KafkaIO Connector)

While trying to access the schema registry using provided (schema url, subject, version, ssl configuration(keystore,truststore..etc))  details , we are getting the below error.

 
Error message from worker: org.apache.kafka.common.errors.GroupAuthorizationException:
Not authorized to access group: initialOffset_offset_consumer_1179967555_kafka-connectivity-test
 
Expected Result : Consumer groupid should not change also should able to connect kafka consumer.
 
Actual Result: 
Though the provided groupid was : kafka-connectivity-test, some how it is changing the value to  initialOffset_offset_consumer_1179967555_kafka-connectivity-test.
 
PFA related code snippets.