[KAFKA-8755] Stand-by Task of an Optimized Source Table Does Not Write Anything to its State Store

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.4.0
• Fix Version/s: 2.4.0
• Component/s: streams
• Labels: newbie
• Assignee: Chris Pettitt
• Reporter: Bruno Cadonna
• Created: 05/Aug/19 22:05
• Updated: 13/Sep/19 22:46
• Resolved: 13/Sep/19 22:46

Description

With the following topology:

builder.table(
    INPUT_TOPIC, 
    Consumed.with(Serdes.Integer(), Serdes.Integer()), 
    Materialized.<Integer, Integer, KeyValueStore<Bytes, byte[]>>as(stateName)
)

and with topology optimization turned on, Kafka Streams uses the input topic INPUT_TOPIC as the change log topic for state store stateName. A stand-by task for such a topology should read from INPUT_TOPIC and should write the records to its state store so that the streams client that runs the stand-by task can take over the execution of the topology in case of a failure with an up-to-date replica of the state.

Currently, the stand-by task described above reads from the input topic but does not write the records to its state store. Thus, after a failure the stand-by task cannot provide any up-to-date state store and the streams client needs to construct the state from scratch before it can take over the execution.

The described behaviour can be reproduced with the attached test.

Issue Links

• links to: GitHub Pull Request #7238