We have IgniteDataStreamer which is used to load data into Ignite under high load. It was previously named IgniteDataLoader, see ticket
See ZeroMQ for more info.
We should create IgniteZeroMqStreamer which will consume messages from Twitter and stream them into Ignite caches.
More details to follow, but to the least we should be able to:
- Convert ZeroMQ messages to Ignite data using an optional pluggable converter. If not provided, we should have some default mechanism.
- Specify the cache name for the Ignite cache to load data into.
- Specify other flags available on IgniteDataStreamer class.