[SAMZA-344] Make task.opts easier to user (part 2)

• Type: Sub-task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: container
• Assignee: Chinmay Soman
• Reporter: Chinmay Soman
• Created: 17/Jul/14 21:00
• Updated: 23/Oct/14 00:52

Description

Address following things in this sub-task:

4. Make ProcessJobFactory behave like YarnJobFactory.

(4) means that ProcessJob and YarnJob should both save a job ID file somewhere (configurable). The ProcessJob's job ID file will contain the PID of the SamzaContainer. The YarnJob's job ID file will contain the application attempt ID for the Samza job in the YARN cluster. We should update kill-job.sh to take a job ID, but to default to a standard location (e.g. bin/../samza-job-name.pid). The YarnJob will use the job ID to kill the job in the YARN cluster. The ProcessJob will use the PID to kill the job locally. We'll need to update the StreamJob interface to support this.