[SAMZA-1465] Performance regression for KafkaCheckpointManager

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.14.0
• Assignee: Jake Maes
• Reporter: Jake Maes
• Created: 17/Oct/17 19:22
• Updated: 18/Oct/17 22:31
• Resolved: 18/Oct/17 22:31

Description

SAMZA-1427 introduced a performance regression with KafkaCheckpointManager.

We invoke the checkpoint manager once for each task and the old code would only read the delta for each subsequent call. The SAMZA-1427 code would reread the entire checkpoint topic for each call.

This effect is even worse when log compaction is not running on the broker.

Issue Links

• links to: GitHub Pull Request #331