Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-3524

Provide a JSONDeserialisationSchema in the kafka connector package

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.1.0
    • Connectors / Kafka

    Description

      (I don't want to include this into 1.0.0)

      Currently, there is no standardized way of parsing JSON data from a Kafka stream. I see a lot of users using JSON in their topics. It would make things easier for our users to provide a serializer for them.

      I suggest to use the jackson library because we have that aready as a dependency in Flink and it allows to parse from a byte[].

      I would suggest to provide the following classes:

      • JSONDeserializationSchema()
      • JSONDeKeyValueSerializationSchema(bool includeMetadata)

      The second variant should produce a record like this:

      {"key": "keydata",
      "value": "valuedata",
      "metadata": {"offset": 123, "topic": "<topic>", "partition": 2 }
      

      Attachments

        Activity

          People

            chesnay Chesnay Schepler
            rmetzger Robert Metzger
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: