[KAFKA-15473] Connect connector-plugins endpoint shows duplicate plugins

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 3.6.0
• Fix Version/s: 3.6.0, 3.7.0
• Component/s: connect
• Assignee: Greg Harris
• Reporter: Greg Harris
• Created: 18/Sep/23 18:48
• Updated: 19/Sep/23 16:58
• Resolved: 19/Sep/23 16:37

Description

In <3.6.0-rc0, duplicates of a plugin would be shown if it subclassed multiple interfaces. For example:

  {
    "class": "org.apache.kafka.connect.storage.StringConverter",
    "type": "converter"
  },
  { 
    "class": "org.apache.kafka.connect.storage.StringConverter",
    "type": "converter"
  },

In 3.6.0-rc0, there are many more listings for the same plugin. For example:

  {
    "class": "org.apache.kafka.connect.storage.StringConverter",
    "type": "converter"
  },
  {
    "class": "org.apache.kafka.connect.storage.StringConverter",
    "type": "converter"
  },
  {
    "class": "org.apache.kafka.connect.storage.StringConverter",
    "type": "converter"
  },
  {
    "class": "org.apache.kafka.connect.storage.StringConverter",
    "type": "converter"
  },
  {
    "class": "org.apache.kafka.connect.storage.StringConverter",
    "type": "converter"
  },
  {
    "class": "org.apache.kafka.connect.storage.StringConverter",
    "type": "converter",
    "version": "3.6.0"
  },

These duplicates appear to happen when a plugin with the same class name appears in multiple locations/classloaders.
When interpreting a connector configuration, only one of these plugins will be chosen, so only one is relevant to show to users. The REST API should only display the plugins which are eligible to be loaded, and hide the duplicates.

Issue Links

• is caused by: KAFKA-15244 Connect PluginType.from(Class) result is incorrect when subclassing multiple plugin interfaces