[SAMZA-1075] Refactor SystemAdmin Interface to expose a common method to create streams.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.13.0
• Assignee: Jake Maes
• Reporter: Jake Maes
• Created: 03/Jan/17 21:20
• Updated: 17/Apr/17 03:36
• Resolved: 17/Feb/17 22:37

Description

With samza-1041, we plan to automatically create intermediate streams for a given graph. To do this, we need a generic way to create streams on all supported systems.

We currently auto-create streams for checkpoints, coordinatorStream, and changelogs, we do so using a fragmented set of interfaces and methods. e.g. to create a checkpoint topic, the KafkaCheckpointMnager.start() creates the topic using KafkaUtil directly, whereas for the coordinatorStream, we call the SystemAdmin.createCoordinatorStream() method. The former is specific to a particular System, but the latter is theoretically (but not actually) implemented for every System.

Instead of using custom methods in differing levels of abstraction, we should just expose one simple createStream() method in the SystemAdmin interface and deprecate the old methods. The new createStream method will receive a complete, system-specific blueprint/specification for the stream and use that to create the actual stream.

Issue Links

• is depended upon by: SAMZA-1041 Multi-stage feature for Samza
• links to: GitHub Pull Request #56
• links to: GitHub Pull Request #57

Sub-Tasks

• Update all existing SystemAdmins to implement the new createStream() method: Resolved