[KAFKA-8554] Generate Topic/Key from Json Transform

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: connect
• Assignee: Unassigned
• Reporter: Gokhan Sari
• Created: 18/Jun/19 14:01
• Updated: 18/Jun/19 14:19

Description

In a configurable pattern, topic and key generation is needed. This pattern could include static values and dynamic parameters which are exist in json tree.

Eg:

• property.format = "signals_{appId}_{date}" >> "signals_app01_18-06-2019"
  • static 'signals'
  • parameter 'appId' from json tree
  • parameter or record 'date'

• property.date.field = "details.signalCreationDate"
  • parameter 'details.signalCreationDate' path from json tree

• property.date.format = "dd-MM-yyyy"
  • date format for date parameters or record dates

 

Extracting topic or key (properties) with these way will led developers to use them according to their business logic.

 

Especially this will be useful for Elasticsearch Kafka Connector in case of dynamic index name and dynamic document id generation needs.

Issue Links

• links to: GitHub Pull Request #6960