[SAMZA-1483] Restart or fail Samza jobs in Zk-based standalone processors when input topic partition changes

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Fix Version/s: 0.15.0
• Assignee: Shanthoosh Venkataraman
• Reporter: Yi Pan
• Created: 07/Nov/17 23:55
• Updated: 11/Jan/18 21:17

Description

This is related to SAMZA-1482. We need to handle input stream partition count change in embedded Samza processors as well.

In Zk-based embedded Samza processors, when a input stream partition count change is detected, we have the opportunity to trigger a JobModel update event, which would:

1. for stateless jobs, restart the embedded SamzaContainer w/ new input topic partitions and new JobModel
2. for stateful jobs, shutdown the embedded StreamProcessor by throwing exceptions when the input topic partitions are different