[HIVE-19878] Hive On Spark support AM shut down when there is no job submit

• Type: New Feature
• Status: Patch Available
• Priority: Minor
• Resolution: Unresolved
• Component/s: Spark
• Assignee: Song Jun
• Reporter: Song Jun
• Created: 13/Jun/18 08:43
• Updated: 05/Aug/18 08:18

Description

the Application Master of Hive on Spark always live on the yarn if the Hive client do not exit(such as one session in HiveServer2), which will accupy lots of resources, we should control the AM shut down when there is no more jobs submit.

Now Tez use the param ` tez.session.am.dag.submit.timeout.secs` to control the DAGAppMaster on the yarn to shut down.

So here Spark need to do this too.

Issue Links

• duplicates: HIVE-14162 Allow disabling of long running job on Hive On Spark On YARN