[KAFKA-6190] GlobalKTable never finishes restoring when consuming transactional messages

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.11.0.1, 1.0.0
• Fix Version/s: 0.11.0.2, 1.0.1, 1.1.0
• Component/s: streams
• Environment: Linux
• Flags: Patch
• External issue URL: https://github.com/apache/kafka/pull/4197
• Assignee: Alex Good
• Reporter: Alex Good
• Created: 08/Nov/17 23:30
• Updated: 09/Nov/17 23:36
• Resolved: 09/Nov/17 23:35

Description

When creating a GlobalKTable from a topic that contains messages that were produced in a transaction the GlobalStreamThread never finishes restoring the table. This appears to be because the `GlobalStateManagerImpl#restoreState` method fails to take into account the transaction markers in it's calculation of it's offset when reading messages and so never reaches the high watermark for the topic it is restoring.

To demonstrate the issue produce a few messages in a transaction to a topic, then attempt to restore a GlobalKTable from that topic, the store will never complete restoring.

Issue Links

• links to: GitHub Pull Request #4197