Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-3988

KafkaConfigBackingStore assumes configs will be stored as schemaless maps

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: 0.10.0.0
    • Fix Version/s: None
    • Component/s: KafkaConnect
    • Labels:
      None

      Description

      If you use an internal key/value converter that drops schema information (as is the default in the config files we provide since we use JsonConverter with schemas.enable=false), the schemas we use that are structs get converted to maps since we don't know the structure to decode them to. Because our tests run with these settings, we haven't validated that the code works if schemas are preserved.

      When they are preserved, we'll hit an error message like this

      [2016-07-25 07:36:34,828] ERROR Found connector configuration (connector-test-mysql-jdbc) in wrong format: class org.apache.kafka.connect.data.Struct (org.apache.kafka.connect.storage.KafkaConfigBackingStore:498)

      because the code currently checks that it is working with a map. We should actually be checking for either a Struct or a Map. This same problem probably affects a couple of other types of data in the same class as Connector configs, Task configs, Connect task lists, and target states are all Structs.

        Attachments

          Activity

            People

            • Assignee:
              ewencp Ewen Cheslack-Postava
              Reporter:
              ewencp Ewen Cheslack-Postava

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 4h
                4h
                Remaining:
                Remaining Estimate - 4h
                4h
                Logged:
                Time Spent - Not Specified
                Not Specified

                  Issue deployment