Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-7620

ConfigProvider is broken for KafkaConnect when TTL is not null

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0, 2.0.1
    • Fix Version/s: 2.2.0, 2.1.1, 2.0.2
    • Component/s: KafkaConnect
    • Labels:
      None

      Description

      If the ConfigData returned by ConfigProvider.get implementations has non-null and non-negative ttl, it will trigger infinite recursion, here is an excerpt of the stack trace:

      at org.apache.kafka.connect.runtime.WorkerConfigTransformer.scheduleReload(WorkerConfigTransformer.java:62)
      	at org.apache.kafka.connect.runtime.WorkerConfigTransformer.scheduleReload(WorkerConfigTransformer.java:56)
      	at org.apache.kafka.connect.runtime.WorkerConfigTransformer.transform(WorkerConfigTransformer.java:49)
      	at org.apache.kafka.connect.runtime.distributed.ClusterConfigState.connectorConfig(ClusterConfigState.java:121)
      	at org.apache.kafka.connect.runtime.distributed.DistributedHerder.connectorConfigReloadAction(DistributedHerder.java:648)
      	at org.apache.kafka.connect.runtime.WorkerConfigTransformer.scheduleReload(WorkerConfigTransformer.java:62)
      	at org.apache.kafka.connect.runtime.WorkerConfigTransformer.scheduleReload(WorkerConfigTransformer.java:56)
      	at org.apache.kafka.connect.runtime.WorkerConfigTransformer.transform(WorkerConfigTransformer.java:49)
      

      Basically,
      1) if a non-null ttl is returned from the config provider, connect runtime will try to schedule a reload in the future,
      2) scheduleReload function reads the config again to see if it is a restart or not, by calling org.apache.kafka.connect.runtime.WorkerConfigTransformer.transform to transform the config
      3) the transform function calls config provider, and gets a non-null ttl, causing scheduleReload being called, we are back to step 1.

      To reproduce, simply fork the provided FileConfigProvider, and add a non-negative ttl to the ConfigData returned by the get functions.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                rayokota Robert Yokota
                Reporter:
                ye Ye Ji
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: