Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-9216

Enforce connect internal topic configuration at startup

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.11.0.0
    • Fix Version/s: 2.3.2, 2.6.0, 2.4.2, 2.5.1
    • Component/s: KafkaConnect
    • Labels:
      None

      Description

      Users sometimes configure Connect's internal topic for configurations with more than one partition. One partition is expected, however, and using more than one leads to weird behavior that is sometimes not easy to spot.

      Here's one example of a log message:

      "textPayload": "[2019-11-20 11:12:14,049] INFO [Worker clientId=connect-1, groupId=td-connect-server] Current config state offset 284 does not match group assignment 274. Forcing rebalance. (org.apache.kafka.connect.runtime.distributed.DistributedHerder:942)\n"
      

      Would it be possible to add a check in the KafkaConfigBackingStore and prevent the worker from starting if connect config partition count !=1 ?

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                EeveeB Evelyn Bayes
                Reporter:
                rhauch Randall Hauch
                Reviewer:
                Randall Hauch
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: