[PIG-3539] Pig should be able to submit multiple DAG

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: tez-branch
• Component/s: tez
• Assignee: Daniel Dai
• Reporter: Daniel Dai
• Created: 23/Oct/13 00:53
• Updated: 21/Nov/14 05:58
• Resolved: 11/Nov/13 19:31

Description

There could be several scenario Pig want to submit multiple DAG:
1. a blocking operator in grunt
2. a blocking operator in script (exec, fs)
3. break a DAG into two for possible inner DAG optimization

Pig should use TezSession so that we can reuse AM. TezSession should be a thread local variable. For those who use Pig Java API, they will use the same AM within a thread, but different AM in another thread.