Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-10555

Improve client state machine

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 2.8.0
    • streams

    Description

      The KafkaStreams client exposes its state to the user for monitoring purpose (ie, RUNNING, REBALANCING etc). The state of the client depends on the state(s) of the internal StreamThreads that have their own states.

      Furthermore, the client state has impact on what the user can do with the client. For example, active task can only be queried in RUNNING state and similar.

      With KIP-671 and KIP-663 we improved error handling capabilities and allow to add/remove stream thread dynamically. We allow adding/removing threads only in RUNNING and REBALANCING state. This puts us in a "weird" position, because if we enter ERROR state (ie, if the last thread dies), we cannot add new threads and longer. However, if we have multiple threads and one dies, we don't enter ERROR state and do allow to recover the thread.

      Before the KIPs the definition of ERROR state was clear, however, with both KIPs it seem that we should revisit its semantics.

      KIP-696: https://cwiki.apache.org/confluence/display/KAFKA/KIP-696%3A+Update+Streams+FSM+to+clarify+ERROR+state+meaning 

      Attachments

        Issue Links

          Activity

            People

              wcarlson5 Walker Carlson
              mjsax Matthias J. Sax
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: