Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-5230

Recommended values for Connect transformations contain the wrong class name

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.10.2.0, 0.10.2.1
    • 0.10.2.2, 0.11.0.0
    • connect

    Description

      If you try to validate a connector config with a transformation, it includes suggested values for that config:

      curl 'http://localhost:8083/connector-plugins/org.apache.kafka.connect.file.FileStreamSourceConnector/config/validate' -X PUT -H 'Content-Type: application/json' -H 'Accept: */*' --data-binary '{"connector.class":"org.apache.kafka.connect.file.FileStreamSourceConnector","name":"blah-blah","transforms":"something","file":"/tmp/blah.what","topic":"test-topic","tasks.max":1}'
      

      However, some of those recommendations do not contain the correct value:

      {
            "definition": {
              "name": "transforms.something.type",
              "type": "CLASS",
              "required": true,
              "default_value": null,
              "importance": "HIGH",
              "documentation": "Class for the 'something' transformation.",
              "group": "Transforms: something",
              "width": "LONG",
              "display_name": "Transformation type for something",
              "dependents": [],
              "order": 0
            },
            "value": {
              "name": "transforms.something.type",
              "value": null,
              "recommended_values": [
                "org.apache.kafka.connect.transforms.ExtractField.Key",
                "org.apache.kafka.connect.transforms.ExtractField.Value",
                "org.apache.kafka.connect.transforms.HoistField.Key",
                "org.apache.kafka.connect.transforms.HoistField.Value",
                "org.apache.kafka.connect.transforms.InsertField.Key",
                "org.apache.kafka.connect.transforms.InsertField.Value",
                "org.apache.kafka.connect.transforms.MaskField.Key",
                "org.apache.kafka.connect.transforms.MaskField.Value",
                "org.apache.kafka.connect.transforms.RegexRouter",
                "org.apache.kafka.connect.transforms.ReplaceField.Key",
                "org.apache.kafka.connect.transforms.ReplaceField.Value",
                "org.apache.kafka.connect.transforms.SetSchemaMetadata.Key",
                "org.apache.kafka.connect.transforms.SetSchemaMetadata.Value",
                "org.apache.kafka.connect.transforms.TimestampRouter",
                "org.apache.kafka.connect.transforms.ValueToKey"
              ],
              "errors": [
                "Missing required configuration \"transforms.something.type\" which has no default value.",
                "Invalid value null for configuration transforms.something.type: Not a Transformation"
              ],
              "visible": true
            }
      

      In particular, nested classes for Key and Value transformations are being returned as, e.g.

      org.apache.kafka.connect.transforms.ReplaceField.Key

      instead of

      org.apache.kafka.connect.transforms.ReplaceField$Key

      It seems this is the difference between the canonical and regular name.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            ewencp Ewen Cheslack-Postava
            ewencp Ewen Cheslack-Postava
            Gwen Shapira Gwen Shapira
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment