[PIG-3602] Tear down TezSessions when Pig exits

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: tez-branch
• Fix Version/s: tez-branch
• Component/s: tez
• Assignee: Rohini Palaniswamy
• Reporter: Cheolsoo Park
• Created: 30/Nov/13 06:36
• Updated: 21/Nov/14 05:58
• Resolved: 20/Dec/13 18:25

Description

Currently, Pig reuses AMs via TezSession, but they are not shut down when Pig exits. There are two problems that I noticed with this-

1. Tez jobs are not marked as finished until TezSessions are expired after timeout. Since they occupy task slots, it blocks submitting jobs.
2. ant clean test-tez leaves orphan processes (DAGAppMaster).

Ideally, TezSession should be kept alive while Pig runs but tore down when Pig exits.