Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-10021

When reading to the end of the config log, check if fetch.max.wait.ms is greater than worker.sync.timeout.ms

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: KafkaConnect
    • Labels:
      None

      Description

      Currently in the Connect code in DistributedHerder.java, we see the following piece of code

       

      {{ if (!canReadConfigs && !readConfigToEnd(workerSyncTimeoutMs))
      return; // Safe to return and tick immediately because readConfigToEnd will do the backoff for us}}

      where the workerSyncTimeoutMs passed in is the timeout given to read to the end of the config log. This is a bug as we should check if fetch.wait.max.ms is greater than worker.sync.timeout.ms and if it is, use worker.sync.timeout.ms as the fetch.wait.max.ms. A better fix would be to use the AdminClient to read to the end of the log, but at a minimum we should check the configs.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              skaundinya Sanjana Kaundinya
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: