Description
Kafka Streams supports an interesting and innovative API for "peeking" into the internal state of running stateful stream processors from outside of the application, called Interactive Query (IQ). This functionality has proven invaluable to users over the years for everything from debugging running applications to serving low latency queries straight from the Streams runtime.
However, the actual interfaces for IQ were designed in the very early days of Kafka Streams, before the project had gained significant adoption, and in the absence of much precedent for this kind of API in peer projects. With the benefit of hindsight, we can observe several problems with the original design that we hope to address in a revised framework that will serve Streams users well for many years to come.
This ticket tracks the implementation of KIP-796: https://cwiki.apache.org/confluence/x/34xnCw
Attachments
Issue Links
- relates to
-
KAFKA-13523 Implement IQv2 support in global stores
- Open
-
KAFKA-13526 IQv2: Consider more generic logic for mapping between binary and typed queries
- Open
-
KAFKA-13541 Make IQv2 query/store interface type safe
- Open
-
KAFKA-13548 IQv2: revisit WindowKeyQuery and WindowRangeQuery
- Open
-
KAFKA-13622 Revisit the complexity of position tracking in state stores
- Open
- links to
1.
|
IQv2: Track Position in KeyValue stores | Closed | Patrick Stuedi | |
2.
|
Implement IQv2 Framework | Closed | John Roesler | |
3.
|
IQ Parity: queries for key/value store range and scan | Resolved | Vicky Papavasileiou | |
4.
|
Session and Window Queries for IQv2 | Resolved | Patrick Stuedi | |
5.
|
IQv2: Track Position in remaining stores | Closed | Patrick Stuedi | |
6.
|
IQv2: Transmit position to standbys | Closed | Vicky Papavasileiou | |
7.
|
IQv2: Implement position tracking and bounding in API | Resolved | John Roesler | |
8.
|
IQv2: Implement KeyQuery from the RecordCache | Resolved | Vicky Papavasileiou | |
9.
|
IQv2: Implement KeyQuery from the KIP | Resolved | John Roesler | |
10.
|
Add PAPI stores to IQv2StoreIntegrationTest | Resolved | John Roesler | |
11.
|
Rename RangeQuery to KeyRangeQuery | Closed | John Roesler | |
12.
|
IQv2: Remove swapResult from the public API | Resolved | John Roesler | |
13.
|
Implement Position restoration for all in-memory state stores | Resolved | Unassigned | |
14.
|
Checkpoint position in state stores | Resolved | Patrick Stuedi |