Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-9314

Connect put() and poll() retries not conforming to KIP-298

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: KafkaConnect
    • Labels:
      None

      Description

      KIP-298 outlines the retry policy of Connect when errors are encountered. In particular, it proposes to retry on RetriableException on put() in SinkTask and poll() in SourceTask.

      However, the code does not reflect this change. For instance, WorkerSourceTask handles RetriableException thrown from poll() by entering into a tight retry loop without backoff. This has led to connectors having to workaround by simply not retrying and failing the task always. Users would need to manually restart the task to recover from even simple network glitches.

      AFAICT from reading code, the same is true for WorkerSinkTask when calling put().

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                nigel.liang Nigel Liang
                Reporter:
                nigel.liang Nigel Liang
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: