Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-3055

JsonConverter mangles schema during serialization (fromConnectData)

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.9.0.0
    • Fix Version/s: 0.9.0.1, 0.10.0.0
    • Component/s: KafkaConnect
    • Labels:
      None

      Description

      Test case is here: https://github.com/ksenji/kafka-connect-test/tree/master/src/test/java/org/apache/kafka/connect/json

      If Caching is disabled, it behaves correctly and JsonConverterWithNoCacheTest runs successfully. Otherwise the test JsonConverterTest fails.

      The reason is that the JsonConverter has a bug where it mangles the schema as it assigns all String fields with the same name (and similar for all Int32 fields)

      This is how the schema & payload gets serialized for the Person Struct (with caching disabled):

      {"schema":{"type":"struct","fields":[{"type":"string","optional":false,"field":"firstName"},{"type":"string","optional":false,"field":"lastName"},{"type":"string","optional":false,"field":"email"},{"type":"int32","optional":false,"field":"age"},{"type":"int32","optional":false,"field":"weightInKgs"}],"optional":false,"name":"Person"},"payload":{"firstName":"Eric","lastName":"Cartman","email":"eric.cartman@southpark.com","age":10,"weightInKgs":40}}
      

      where as when caching is enabled the same Struct gets serialized as (with caching enabled) :

      {"schema":{"type":"struct","fields":[{"type":"string","optional":false,"field":"email"},{"type":"string","optional":false,"field":"email"},{"type":"string","optional":false,"field":"email"},{"type":"int32","optional":false,"field":"weightInKgs"},{"type":"int32","optional":false,"field":"weightInKgs"}],"optional":false,"name":"Person"},"payload":{"firstName":"Eric","lastName":"Cartman","email":"eric.cartman@southpark.com","age":10,"weightInKgs":40}}
      

      As we can see all String fields became "email" and all int32 fields became "weightInKgs".

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                ewencp Ewen Cheslack-Postava
                Reporter:
                ksenji Kishore Senji
                Reviewer:
                Ewen Cheslack-Postava
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: