Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-9274

Gracefully handle timeout exceptions on Kafka Streams

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.8.0
    • streams

    Description

      Right now streams don't treat timeout exception as retriable in general by throwing it to the application level. If not handled by the user, this would kill the stream thread unfortunately.

      In fact, timeouts happen mostly due to network issue or server side unavailability. Hard failure on client seems to be an over-kill.

      We would like to discuss what's the best practice to handle timeout exceptions on Streams. The current state is still brainstorming and consolidate all the cases that contain timeout exception within this ticket.

      This ticket is backed by KIP-572: https://cwiki.apache.org/confluence/display/KAFKA/KIP-572%3A+Improve+timeouts+and+retries+in+Kafka+Streams

      Attachments

        Issue Links

          causes

          Bug - A problem which impairs or prevents the functions of the product. KAFKA-12951 Infinite loop while restoring a GlobalKTable

          • Major - Major loss of function.
          • Resolved
          contains

          Bug - A problem which impairs or prevents the functions of the product. KAFKA-9270 KafkaStream crash on offset commit failure

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. KAFKA-8311 Better consumer timeout exception handling

          • Major - Major loss of function.
          • Resolved
          incorporates

          Improvement - An improvement or enhancement to an existing feature or task. KAFKA-7932 Streams needs to handle new Producer exceptions

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. KAFKA-8858 Kafka Streams - Failed to Rebalance Error and stream consumer stuck for some reason

          • Major - Major loss of function.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. KAFKA-8803 Stream will not start due to TimeoutException: Timeout expired after 60000milliseconds while awaiting InitProducerId

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link GitHub Pull Request #8060

          Web Link GitHub Pull Request #8120

          Web Link GitHub Pull Request #8122

          Web Link GitHub Pull Request #8864

          Web Link GitHub Pull Request #9047

          Web Link GitHub Pull Request #9060

          Web Link GitHub Pull Request #9333

          Web Link GitHub Pull Request #9368

          Web Link GitHub Pull Request #9385

          Web Link GitHub Pull Request #9570

          Web Link GitHub Pull Request #9800

          Web Link GitHub Pull Request #9997

          Web Link GitHub Pull Request #10000

          Web Link GitHub Pull Request #10072

          Web Link GitHub Pull Request #10374

          mentioned in

          Page Loading...

          Page Loading...

          Activity

            People

              mjsax Matthias J. Sax
              bchen225242 Boyang Chen
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: