Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-8713

[Connect] JsonConverter NULL Values are replaced by default values even in NULLABLE fields

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.0.0, 2.0.1, 2.1.0, 2.2.0, 2.3.0, 2.2.1
    • 3.5.0
    • connect

    Description

      Class JsonConverter line: 582

          private static JsonNode convertToJson(Schema schema, Object logicalValue) {
              if (logicalValue == null) {
                  if (schema == null) // Any schema is valid and we don't have a default, so treat this as an optional schema
                      return null;
                  if (schema.defaultValue() != null)
                      return convertToJson(schema, schema.defaultValue());
                  if (schema.isOptional())
                      return JsonNodeFactory.instance.nullNode();
                  throw new DataException("Conversion error: null value for field that is required and has no default value");
              }
              ....
          }
      

      Expect:

      Value `null` is valid for an optional filed, even though the filed has a default value.
      Only when field is required, the converter return default value fallback when value is `null`.

      Actual:

      Always return default value if `null` was given.

      Example:

      I'm not sure if the current behavior is the exactly expected, but at least on MySQL, a table define as

      create table t1 {
         name varchar(40) not null,
         create_time datetime default '1999-01-01 11:11:11' null,
         update_time datetime default '1999-01-01 11:11:11' null
      }
      

      Just insert a record:

      INSERT INTO `t1` (`name`,  `update_time`) VALUES ('kafka', null);
      

      The result is:

      {
          "name": "kafka",
          "create_time": "1999-01-01 11:11:11",
          "update_time": null
      }
      

      But when I use debezium pull binlog and send the record to Kafka with JsonConverter, the result changed to:

      {
          "name": "kafka",
          "create_time": "1999-01-01 11:11:11",
          "update_time": "1999-01-01 11:11:11"
      }
      

      For more details, see: https://issues.jboss.org/browse/DBZ-1064

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            mimaison Mickael Maison
            chengpan Cheng Pan
            Votes:
            1 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment