[SAMZA-1212] Refactor interaction between StreamProcessor, JobCoordinator and SamzaContainer

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.13.0
• Assignee: Navina Ramesh
• Reporter: Navina Ramesh
• Created: 13/Apr/17 23:57
• Updated: 03/May/17 22:10
• Resolved: 03/May/17 22:10

Description

Currently, StreamProcessor is a dummy wrapper for JobCoordinator and the JobCoordinator directly controls the lifecycle of SamzaContainer. This has a couple of drawbacks:
1. Each implementation of JobCoordinator needs to manage the container lifecycle. This seems redundant as the lifecycle of SamzaContainer is driven by availability of JobModel and updates to it.
2. The StreamProcessor interface requirements kind of bleed into JobCoordinator interface. For example, StreamProcessor interacts with user-api. If the requirements for the user-api change , it will directly impact the JobCoordinator interface as well. A good example of such a design creep is the "awaitStop" interface. While this is a reasonable requirement from the user's perspective of a StreamProcessor, it should not impose the same requirement on every implementation of JobCoordinator.
3. Error handling is pretty straight-forward when there is a control hierarchy of components. However, some implementations of JobCoordinators are synchronous, while others are Asynchronous. This will directly impact how errors/exceptions are propagated across the stack. It will be much cleaner if the StreamProcessor can handle the errors/exception from both container and coordinator.

These are the primary motivations for refactoring.

Issue Links

• is depended upon by: SAMZA-1064 Standalone Samza with Zookeeper for Coordination
• links to: GitHub Pull Request #148