[KAFKA-13028] AbstractConfig should allow config provider configuration to use variables referencing other config providers earlier in the list

• Type: Improvement
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Component/s: clients, connect
• Assignee: Randall Hauch
• Reporter: Randall Hauch
• Created: 02/Jul/21 18:04
• Updated: 07/Jul/21 20:32

Description

When AbstractConfig recognizes config provider properties, it instantiates all of the config providers first and then uses those config providers to resolve any variables in remaining configurations. This means that if you define two config providers with:

config.providers=providerA,providerB
...

then the configuration properties for the second provider (e.g., `providerB`) cannot use variables that reference the first provider (e.g., `providerA`). In other words, this is not possible:

config.providers=providerA,providerB
config.providers.providerA.class=FileConfigProvider
config.providers.providerB.class=ComplexConfigProvider
config.providers.providerA.param.client.key=${file:/usr/secrets:complex.client.key}
config.providers.providerA.param.client.secret=${file:/usr/secrets:complex.client.secret}

This should be possible if the config providers are instantiated and configured in the same order as they appear in the `config.providers` property. The benefit is that it allows another level of indirection so that any secrets required by config provider can be resolved using an earlier simple config provider.

For example, config providers are often defined in Connect worker configurations to resolve secrets within connector configurations, or to resolve secrets within the worker configuration itself (e.g., producer or consumer secrets). But it would be useful to also be able to resolve the secrets needed by one configuration provider using another configuration provider that is defined earlier in the list.

Issue Links

• links to: GitHub Pull Request #10961