[SAMZA-364] Refactor lifecycle listener APIs

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.8.0
• Component/s: container
• Assignee: Unassigned
• Reporter: Chris Riccomini
• Created: 01/Aug/14 22:47
• Updated: 07/Nov/14 01:24

Description

There are a many cases where a developer might like to plug into different points throughout the lifecycle of a Samza job's execution. Example use cases for this include:

• Wanting to initialize setup/tear down of a slf4j log binding (SAMZA-350)
• Wanting to initialize remote systems that the job interacts with.
• Wanting to setup a custom HTTP (or some other) server for operational purposes.

Potential injection points include:

1. SamzaContainer startup/shutdown.
2. Before/after processing a message in a TaskInstance.
3. Before/after init'ing a TaskInstance.
4. Before/after shutting down a TaskInstance.
5. Before/after windowing a TaskInstance.
6. Before/after committing offsets.
7. Before/after a job is started.
8. All of the YARN AM related events.

Right now, we have a TaskLifecycleListener. This is insufficient to cover all cases, and is somewhat cumbersome to deal with because we instantiate one listener per TaskInstance in a container.

We should refactor and re-design the lifecycle APIs for Samza. This will include investigating how other servlets/containers handle things, writing a doc, and implementing the new API.

I expect that this will result in backwards incompatible changes with TaskLifecycleListener. I think this is OK.

Issue Links

• relates to: SAMZA-437 Remove TaskLifecycleListener