Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-13096

QueryableStoreProvider is not updated when threads are added/removed/replaced rendering IQ impossible

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • 2.8.0
    • 2.8.1, 3.0.0
    • streams
    • None

    Description

      The QueryableStoreProviders class is used to route queries to the correct state store on the owning StreamThread, making it a critical piece of IQ. It gets instantiated when you create a new KafkaStreams, and is passed in a list of StreamThreadStateStoreProviders which it then copies and stores. Because it only stores a copy it only ever contains a provider for the StreamThreads that were created during the app's startup, and unfortunately is never updated during an add/remove/replace thread event. 

      This means that IQ can’t get a handle on any stores that belong to a thread that wasn’t in the original set. If the app is starting up new threads through the #addStreamThread API or following a REPLACE_THREAD event, none of the data in any of the stores owned by that new thread will be accessible by IQ. If a user is removing threads through #removeStreamThread, or threads die and get replaced, you can fall into an endless loop of InvalidStateStoreException from doing a lookup into stores that have been closed since the thread was removed/died.

      If over time all of the original threads are removed or replaced, then IQ won’t work at all.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              ableegoldman A. Sophie Blee-Goldman
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: