Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-5156 Options for handling exceptions in streams
  3. KAFKA-5157

Options for handling corrupt data during deserialization

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.0.0
    • streams

    Description

      When there is a bad formatted data in the source topics, deserialization will throw a runtime exception all the way to the users. And since deserialization happens before it was ever processed at the beginning of the topology, today there is no ways to handle such errors on the user-app level.
      We should consider allowing users to handle such "poison pills" in a customizable way.

      https://cwiki.apache.org/confluence/display/KAFKA/KIP-161%3A+streams+deserialization+exception+handlers

      Attachments

        Issue Links

          is blocked by

          Bug - A problem which impairs or prevents the functions of the product. KAFKA-4740 Using new consumer API with a Deserializer that throws SerializationException can lead to infinite loop

          • Critical - Crashes, loss of data, severe memory leak.
          • Open
          is part of

          Sub-task - The sub-task of the issue KAFKA-5301 Improve exception handling on consumer path

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link GitHub Pull Request #3423

          mentioned in

          Page Loading...

          Activity

            People

              enothereska Eno Thereska
              enothereska Eno Thereska
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: