[KAFKA-8340] ServiceLoader fails when used from isolated plugin path directory

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.3.0
• Fix Version/s: 1.0.3, 1.1.2, 2.0.2, 2.1.2, 2.2.2, 2.4.0, 2.5.0, 2.3.2
• Component/s: connect
• Assignee: Greg Harris
• Reporter: Chris Egerton
• Reviewer: Randall Hauch
• Created: 08/May/19 21:26
• Updated: 12/Feb/24 22:29
• Resolved: 17/Oct/19 02:14

Description

Under some circumstances, the ServiceLoader.load mechanism will fail when used from an isolated plugin path directory and return an incomplete (often empty) ServiceLoader instance.

 

To replicate:

• Include a META-INF/services/... file in one of the JARS located in a plugin's directory with one or more implementations of that service listed inside. For the sake of example, let's say the name of this service is com.example.MyService
• Program that plugin to invoke ServiceLoader.load(com.example.MyService.class)
• Start the Connect framework, making sure this plugin is included on the plugin path and that it somehow invokes the ServiceLoader.load(...) method
• Observe that the services loaded by that invocation do not include the ones described in the META-INF/services/... file contained in the JAR in the plugin's directory

 

This is because the ServiceLoader.load(Class) method uses the current thread's context classloader to locate resources and load services. The current thread's context classloader is, in most cases, an instance of DelegatingClassLoader, which will (unless asked to locate resources corresponding to a provider-configuration file for a REST extension or config provider) simply delegate resource location to the parent and, unless asked to locate a class for a recognized plugin, also delegate class loading to the parent. Thus, none of the plugin's JARs are scanned for either provider-configuration files or for actual service classes.

A viable workaround for some cases is to instead use the ServiceLoader.load(Class, ClassLoader) method, specifying the current class's classloader as the second argument. This causes the plugin's PluginClassLoader, which will scan all JARs in the plugin's directory to be used to locate resources and classes.

However, this may not be feasible in all cases, especially when working with external libraries that may be difficult or impossible to apply this workaround on.

Issue Links

• fixes: KAFKA-6007 Connect can't validate against transforms in plugins.path
• is related to: KAFKA-8819 Plugin path for converters not working as intended
• is related to: KAFKA-14654 Connectors have incorrect Thread Context Classloader during initialization
• is related to: KAFKA-9376 Plugin class loader not found using MM2
• is related to: KAFKA-14670 Refactor connect plugins to be called from wrapper classes
• relates to: KAFKA-9057 Backport KAFKA-8819 and KAFKA-8340 to 1.1 and 1.0 branches
• links to: GitHub Pull Request #7315