[APEXMALHAR-2158] Duplication of data emitted when the Kafka Input Operator(0.8 version) redeploys

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.5.0
• Assignee: Chaitanya Chebolu
• Reporter: Chaitanya Chebolu
• Created: 18/Jul/16 11:07
• Updated: 04/Sep/16 01:40
• Resolved: 18/Aug/16 06:01

Description

Kafka Input Operator failure recovery depends on offsetStats map.
"offsetStats" map is updated whenever the message is emitted from operator.

When the operator goes down and comes back, operator sets this map to the consumer and start reading from these offsets. Consumer reads the last emitted message again and operator emits it.

Solution: Increment the offsets before setting to the consumer.

Issue Links

• links to: GitHub Pull Request #346