[KAFKA-12316] Configuration is not defined: topic.creation.default.partitions

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.6.1
• Component/s: connect
• Assignee: Unassigned
• Reporter: Goltseva Taisiia
• Created: 09/Feb/21 11:41
• Updated: 09/Feb/21 11:49

Description

Hi, guys!

The KIP was implemented:

https://cwiki.apache.org/confluence/display/KAFKA/KIP-158%3A+Kafka+Connect+should+allow+source+connectors+to+set+topic-specific+settings+for+new+topics

 

But it seems you forget to add changes to the class:

https://github.com/apache/kafka/blob/trunk/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/AbstractHerder.java

 

I suppose we need to add something like 'validateClientOverrides()' for configs starting with 'topic.creation' prefix. Like this:

https://github.com/apache/kafka/blob/trunk/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/AbstractHerder.java#L420

 

For now when I create, for example, a Postgres Source connector and do not specify login and password (it's mandatory parameters) I get this:

{    "error_code": 400,    "message": "Connector configuration is invalid and contains the following 1 error(s):\nConfiguration is not defined: topic.creation.default.partitions\nConfiguration is not defined: topic.creation.test1.retention.ms\nConfiguration is not defined: topic.creation.test1.include\nConfiguration is not defined: topic.creation.test1.partitions\nConfiguration is not defined: topic.creation.default.replication.factor\nA value is required\nYou can also find the above list of errors at the endpoint `/connector-plugins/{connectorType}/config/validate`"}

But it should be just:
 

{     "error_code": 400,     "message": "Connector configuration is invalid and contains the following 1 error(s):\nA value is required\nYou can also find the above list of errors at the endpoint `/connector-plugins/{connectorType}/config/validate`" }

So, I think, a little change of AbstractHerder class is required.
 

Issue Links

• is related to: KAFKA-5295 Allow Kafka Connect source connectors to specify topic-specific settings for new topics