Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
0.10.0
-
None
-
None
Description
Currently we have a task that may produce a message that is too large to be sent to Kafka. In the current KafkaSystemProducer a MessageSizeTooLargeException is thrown that we can't catch.
Configuring Kafka to accept a message of the correct size works well but we would like to prevent a StreamTask from entering a retry loop since it will continue to fail each time it produces that super large message.
We would like to be able to configure which exceptions trigger a shutdown or merely dropping a message on the floor with a large warning. We are also open to other suggestions on what the proper method for properly handling that exception.