Details
-
Sub-task
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
0.10.0.0
Description
As for 0.10.0.0, the worker threads (i.e. StreamThreads) can possibly encounter the following runtime exceptions:
1) consumer.poll() could throw KafkaException if some of the configuration are not accepted, such as topics not authorized to read / write (security), session-timeout value not valid, etc; these exceptions will be thrown in the first ever poll().
2) task.addRecords() could throw KafkaException (most likely SerializationException) if the deserialization fails.
3) task.process() / punctuate() could throw various KafkaException; for example, serialization / deserialization errors, state storage operation failures (RocksDBException, for example), producer sending failures, etc.
4) maybeCommit / commitAll / commitOne could throw various Exceptions if the flushing of state store fails, and when consumer.commitSync throws exceptions other than CommitFailedException.
For all the above 4 cases, KafkaStreams does not capture and handle them, but expose them to users, and let users to handle them via KafkaStreams.setUncaughtExceptionHandler. We need to re-think if the library should just handle these cases without exposing them to users and kill the threads / migrate tasks to others since they are all not recoverable.
Attachments
Issue Links
- links to
