[TEZ-340] [Umbrella] Support Tez Sessions

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.2.0
• Assignee: Hitesh Shah
• Reporter: Bikas Saha
• Created: 06/Aug/13 02:02
• Updated: 03/Jan/14 19:28
• Resolved: 03/Jan/14 19:28

Description

The DAG AM currently expects the DAG plan to be available during AM launch. Enhancing this to allow more DAG's to be sent to a currently running AM increases the possibility of re-using more resources and allows drill down queries to be made more efficient. This is not intended to be a service.

Sub-Tasks

• Allow DAG to be submitted to an AM after it has started: Closed
• Time out AM after a certain period if no DAG submitted: Closed
• Add initial support for Tez Sessions: Closed
• Decouple AM state from DAG state when handling multiple dags.: Resolved
• Cleanup handling of SESSION info being passed in via env: Resolved
• Add support for getting status of a tez session: Closed
• Add support for multiple DAGs within the same AM: Closed
• Enforce a timeout on the client when submitting to a tez session: Closed
• job sometimes fails on launch when using session mode: Closed
• Add support for session jars and pre-warming of containers to be re-used across a session: Closed
• Fix logging in pre-warmed containers: Closed
• Support simple container re-use across DAGs within a session: Closed