[IGNITE-428] Implement IgniteKafkaStreamer to stream data from Apache Kafka

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: sprint-5
• Fix Version/s: sprint-7
• Component/s: streaming
• Assignee: Yakov Zhdanov
• Reporter: Dmitriy Setrakyan
• Created: 08/Mar/15 03:03
• Updated: 03/Jul/15 17:05
• Resolved: 03/Jul/15 17:05

Description

We have IgniteDataStreamer which is used to load data into Ignite under high load. It was previously named IgniteDataLoader, see ticket IGNITE-394.

See Apache Kafka for more information.

We should create IgniteKafkaStreamer which will consume messages from Kafka and stream them into Ignite caches.

More details to follow, but to the least we should be able to:

• Convert Kafka messages to Ignite data using an optional pluggable converter.
• Specify the cache name for the Ignite cache to load data into.
• Specify other flags available on IgniteDataStreamer class.