[KAFKA-8858] Kafka Streams - Failed to Rebalance Error and stream consumer stuck for some reason

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 2.1.1
• Component/s: streams
• Labels: Stream consumer corrupt offset rebalance transactions
• Environment: Apache Kafka 2.1.1
• Assignee: Guozhang Wang
• Reporter: Ante B.
• Created: 02/Sep/19 13:25
• Updated: 26/Jan/21 20:19
• Resolved: 06/May/20 18:49

Description

I have a basic Kafka Streams application that reads from a topic, performs a rolling aggregate, and performs a join to publish to an agg_topic. Our project has the timeout failure in Kafka 2.1.1 env and we don't know the reason yet.

Our stream consumer stuck for some reason. 

After we changed our group id to another one it became normal. So seems offset data for this consumer is corrupted.

Can you help us please to resolve this problem to be able to revert us to the previous consumer name because we have many inconveniences due to this.

Ping me pls if you will need some additional info.

Our temporary workaround is to disable the exactly_once config which skips the initializing transactional state. Also offset reseted for corrupted partition, with no effect.

Full problem description in log:

[2019-08-30 14:20:02.168] [abc-streamer-StreamThread-21] ERROR org.apache.kafka.streams.processor.internals.StreamThread:273 - stream-thread [abc-streamer-StreamThread-21] Error caught during partition assignment, will abort the current process and re-throw at the end of rebalance: {}
org.apache.kafka.common.errors.TimeoutException: Timeout expired while initializing transactional state in 60000ms.
[2019-08-30 14:21:35.407] [abc-streamer-StreamThread-14] ERROR org.apache.kafka.streams.processor.internals.StreamThread:273 - stream-thread [abc-streamer-StreamThread-14] Error caught during partition assignment, will abort the current process and re-throw at the end of rebalance: {}
org.apache.kafka.common.errors.TimeoutException: Timeout expired while initializing transactional state in 60000ms.
[2019-08-30 14:22:58.487] [abc-streamer-StreamThread-13] ERROR org.apache.kafka.streams.processor.internals.StreamThread:273 - stream-thread [abc-streamer-StreamThread-13] Error caught during partition assignment, will abort the current process and re-throw at the end of rebalance: {}
org.apache.kafka.common.errors.TimeoutException: Timeout expired while initializing transactional state in 60000ms.

 

 

 

Issue Links

• duplicates: KAFKA-8803 Stream will not start due to TimeoutException: Timeout expired after 60000milliseconds while awaiting InitProducerId
• relates to: KAFKA-9274 Gracefully handle timeout exceptions on Kafka Streams