[KAFKA-7932] Streams needs to handle new Producer exceptions

• Type: Improvement
• Status: Resolved
• Priority: Critical
• Resolution: Duplicate
• Component/s: streams
• Assignee: John Roesler
• Reporter: John Roesler
• Created: 14/Feb/19 19:12
• Updated: 27/Feb/20 21:51
• Resolved: 27/Feb/20 21:51

Description

Following on KAFKA-7763, Streams needs to handle the new behavior.

See also https://github.com/apache/kafka/pull/6066

Streams code (StreamTask.java) needs to be modified to handle the new exception.

From the upstream change, `commtit/abort Transaction` can also throw TimeoutException now: default `MAX_BLOCK_MS_CONFIG` in producer is 60 seconds, so I think just wrapping it as StreamsException should be reasonable, similar to what we do for `producer#send`'s TimeoutException (https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/processor/internals/RecordCollectorImpl.java#L220-L225 ).

 

See also https://github.com/apache/kafka/pull/6066#issuecomment-464403448

Issue Links

• is cloned by: KAFKA-8040 Streams needs to handle timeout in initTransactions
• is part of: KAFKA-9274 Gracefully handle timeout exceptions on Kafka Streams
• is related to: KAFKA-6446 KafkaProducer with transactionId endless waits when bootstrap server is down
• is related to: KAFKA-6127 Streams should never block infinitely
• is related to: KAFKA-7763 KafkaProducer with transactionId endless waits when network is disconnection for 10-20s