[KAFKA-17117] Avoid instantiating classpath plugins when service loading plugins

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 3.6.0
• Component/s: connect
• Assignee: Unassigned
• Reporter: Greg Harris
• Created: 10/Jul/24 21:10
• Updated: 11/Jul/24 15:36

Description

In KAFKA-14789 modifications were made to allow PluginClassLoaders to see all resource files of the parent DelegatingClassLoader and classpath, rather than selectively hiding some resources that were for ServiceLoader manifests.

This has the effect that the ServiceLoader finds classpath plugins when searching in plugin locations, and the PluginScanner filters these plugins out by checking for classloader equality.

This has some side-effects that are undesirable:

• Classpath plugins may be instantiated with the thread context classloader set to a plugin classloader
• Classpath plugins are instantiated multiple times, once for each plugin location
• Exceptions from classpath plugins show up multiple times in the logs: KAFKA-17111

This change may require us to fork the ServiceLoader implementation, which is itself undesirable.

Issue Links

• is caused by: KAFKA-14789 Plugin ServiceLoader visibility from isolated plugins is inconsistent
• is related to: KAFKA-17111 ServiceConfigurationError in JsonSerializer/Deserializer during Plugin Discovery