Details
-
Improvement
-
Status: Reopened
-
Major
-
Resolution: Unresolved
-
None
-
None
Description
From Guozhang Wang:
"Only default serdes provided through configs are auto-configured today. But we could auto-configure other serdes passed alongside the topology builder as well."
After the PR was merged, we realized that the current approach to implement this features is actually not backward compatible. Thus, we need to revert the commit for now to not break backward compatibility in 2.3 release. After some more thinking, it seems that this feature is actually more complicated to get right as it seem on the surface and hence it would required a proper KIP.
The following issues are identified:
- in the new code, configure() would be called twice, one in user code (if people don't rewrite existing applications) and later via Kafka Streams – the second call could "reconfigure" the Serde and overwrite the correct configuration from the first call done by the user
- if there are multiple Serdes using the same configuration parameters names, it's only possible to specify this parameter name once in the global StreamsConfig; hence, it's not possible for users to configure both Serdes differently
- basically, the global StreamsConfig needs to contain all configuration parameters over all used Serdes to make a potential second call to `configure()` idempotant
To address the issues, some ideas would be:
- pass in the configuration via the constructor and deprecate `configure()` method
- add a new method `isConfigured()` that would allow to skip the second configuration call within Kafka Streams runtime
There might be other ways to address this, and the different options should be discussed on the KIP.