Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-9274

Gracefully handle timeout exceptions on Kafka Streams

Agile BoardAttach filesAttach ScreenshotVotersStop watchingWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.8.0
    • streams

    Description

      Right now streams don't treat timeout exception as retriable in general by throwing it to the application level. If not handled by the user, this would kill the stream thread unfortunately.

      In fact, timeouts happen mostly due to network issue or server side unavailability. Hard failure on client seems to be an over-kill.

      We would like to discuss what's the best practice to handle timeout exceptions on Streams. The current state is still brainstorming and consolidate all the cases that contain timeout exception within this ticket.

      This ticket is backed by KIP-572: https://cwiki.apache.org/confluence/display/KAFKA/KIP-572%3A+Improve+timeouts+and+retries+in+Kafka+Streams

      Attachments

        Issue Links

        causes

        Bug - A problem which impairs or prevents the functions of the product. KAFKA-12951 Infinite loop while restoring a GlobalKTable

        • Major - Major loss of function.
        • Resolved
        contains

        Bug - A problem which impairs or prevents the functions of the product. KAFKA-9270 KafkaStream crash on offset commit failure

        • Critical - Crashes, loss of data, severe memory leak.
        • Resolved

        Improvement - An improvement or enhancement to an existing feature or task. KAFKA-8311 Better consumer timeout exception handling

        • Major - Major loss of function.
        • Resolved
        incorporates

        Improvement - An improvement or enhancement to an existing feature or task. KAFKA-7932 Streams needs to handle new Producer exceptions

        • Critical - Crashes, loss of data, severe memory leak.
        • Resolved
        is related to

        Bug - A problem which impairs or prevents the functions of the product. KAFKA-8858 Kafka Streams - Failed to Rebalance Error and stream consumer stuck for some reason

        • Major - Major loss of function.
        • Resolved
        relates to

        Bug - A problem which impairs or prevents the functions of the product. KAFKA-8803 Stream will not start due to TimeoutException: Timeout expired after 60000milliseconds while awaiting InitProducerId

        • Major - Major loss of function.
        • Resolved
        links to

        Web Link GitHub Pull Request #8060

        Web Link GitHub Pull Request #8120

        Web Link GitHub Pull Request #8122

        Web Link GitHub Pull Request #8864

        Web Link GitHub Pull Request #9047

        Web Link GitHub Pull Request #9060

        Web Link GitHub Pull Request #9333

        Web Link GitHub Pull Request #9368

        Web Link GitHub Pull Request #9385

        Web Link GitHub Pull Request #9570

        Web Link GitHub Pull Request #9800

        Web Link GitHub Pull Request #9997

        Web Link GitHub Pull Request #10000

        Web Link GitHub Pull Request #10072

        Web Link GitHub Pull Request #10374

        mentioned in

        Page Loading...

        Page Loading...

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            mjsax Matthias J. Sax
            bchen225242 Boyang Chen
            Votes:
            0 Vote for this issue
            Watchers:
            8 Stop watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment