Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
Description
This is related to SAMZA-1482. We need to handle input stream partition count change in embedded Samza processors as well.
In Zk-based embedded Samza processors, when a input stream partition count change is detected, we have the opportunity to trigger a JobModel update event, which would:
- for stateless jobs, restart the embedded SamzaContainer w/ new input topic partitions and new JobModel
- for stateful jobs, shutdown the embedded StreamProcessor by throwing exceptions when the input topic partitions are different