Description
1. Since KIP-285 (defining the basic auth extension) the classloading isolation mechanism has used ServiceLoader to find rest extensions.
2. Since KIP-297 (adding secrets externalization via ConfigProvider) serviceloading was used to discover ConfigProviders.
3. It was noticed that the isolation mechanism improperly discovered service loaded plugins and attributed classpath plugins to all isolated plugins. This was fixed by KAFKA-6991, in which the ConnectRestExtension and ConfigProvider manifest files are hidden from the isolated plugins, in order to hide them from the scanning ServiceLoader calls.
4. Since KIP-458 (adding ConnectorConfigOverridePolicy) serviceloading was used to discover ConnectorConfigOverridePolicy instances, but these manifests were not hidden from plugin classloaders. ConnectorConfigOverridePolicy objects are currently mis-attributed, but this has had no ill-effects at this time.
5. With KIP-898, all plugins will be loaded with the ServiceLoader, and so all other plugins could potentially encounter this mis-attribution bug that was only resolved for 2 of the plugins in the past.
The current implementation relies on a string equality test to deny reading the serviceloader manifests that cause mis-attribution, which is a brittle solution. It was very easy for the person implementing KIP-458 to forget to add the new class to the manifest denylist, and cause a re-appearance of the mis-attribution bug.
The denylist approach to preventing mis-attribution also changes the visibility of classpath plugins from isolated classloaders. We should choose whether the visibility of classpath plugins is desirable, and either eliminate the denylist or make it more difficult to get out-of-sync with the current list of plugins.
Attachments
Issue Links
- causes
-
KAFKA-17117 Avoid instantiating classpath plugins when service loading plugins
-
- Open
-
- is related to
-
KAFKA-14627 Modernize Connect plugin discovery
-
- Resolved
-
- links to