[STORM-810] PartitionManager in storm-kafka should commit latest offset before close

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.10.0, 1.0.0
• Fix Version/s: 0.10.0
• Component/s: storm-kafka
• Assignee: xiajun
• Reporter: xiajun
• Created: 06/May/15 08:05
• Updated: 08/Jul/15 02:35
• Resolved: 01/Jul/15 13:44

Description

When KafkaSpout refresh with kafka partitions, it will remove patitions that no longer served by this spout and close the connection. But before close the connection, PartitionManager don't commit the latest offset. As a result some other spout that serve the removed partition will replay some tuple that between last commit and close.