[UNOMI-66] Integrate Apache Kafka

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: unomi-1.5.0
• Component/s: unomi(-core)
• Assignee: Jean-Baptiste Onofré
• Reporter: Serge Huber
• Created: 22/Nov/16 10:30
• Updated: 11/Apr/23 07:30
• Resolved: 21/Apr/20 11:13

Description

It would be really great to integrate Apache Kafka into Apache Unomi so that we can push event data and other "heavy" operations into its messaging systems and therefore make it easier to decouple some large scale operations that might require a lot of processing power such as large segment updates, etc...

An interesting idea would be to put Kafka in "front" of Unomi to absorb the traffic and then clients could use something like the Kafka REST Proxy  to insert events into Unomi.

This would also make it a lot easier to integrate with BigData systems such as Apache Spark, Hadoop or others.

We might want at the same time to see if we can define a mechanism to get the status of long-running operations.

Issue Links

• links to: GitHub Pull Request #99