[KAFKA-10500] Add API to Start and Stop Stream Threads

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.8.0
• Component/s: streams
• Labels: kip
• Assignee: Walker Carlson
• Reporter: Bruno Cadonna
• Created: 18/Sep/20 09:47
• Updated: 03/Feb/21 01:14
• Resolved: 29/Jan/21 20:51

Description

Currently, there is no possibility in Kafka Streams to increase or decrease the number of stream threads after the Kafka Streams client has been started. Uncaught exceptions thrown in a stream thread kill the stream thread leaving the Kafka Streams client with less stream threads for processing than when the client was started. The only way to replace the killed stream thread is to restart the whole Kafka Streams client. For transient errors, it might make sense to replace a killed stream thread with a new one while users try to find the root cause of the error. That could be accomplished by starting a new stream thread in the uncaught exception handler of the killed stream thread.

Part of KIP-663 https://cwiki.apache.org/confluence/display/KAFKA/KIP-663%3A+API+to+Start+and+Shut+Down+Stream+Threads 

Issue Links

• relates to: KAFKA-10015 React Smartly to Unexpected Errors on Stream Threads
• links to: GitHub Pull Request #9543
• links to: GitHub Pull Request #9572
• links to: GitHub Pull Request #9581
• links to: GitHub Pull Request #9614
• links to: GitHub Pull Request #9615
• links to: GitHub Pull Request #9647
• links to: GitHub Pull Request #9695
• links to: GitHub Pull Request #9974
• mentioned in: Page Loading...