[IGNITE-16871] Eliminate periodically meta update with marker records in event topic

• Type: Sub-task
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Labels: IEP-59
• Ignite Flags: Docs Required, Release Notes Required
• Assignee: Nikolay Izhikov
• Reporter: Nikolay Izhikov
• Created: 18/Apr/22 13:22
• Updated: 17/May/22 06:55
• Resolved: 17/May/22 06:55

Description

Current solution for KafkaToIgniteStreamer implements periodic update approach.
It seems we can eliminate periodic updates with the following:

1. Each time new meta update found - marker record created in each partition of events topic.

2. If meta record read from events topic then KafkaToIgniteStreamer must read all new meta from meta topic.

Issue Links

• links to: GitHub Pull Request #141
• links to: PR #141