[HIVE-9449] Push YARN configuration to Spark while deply Spark on YARN [Spark Branch]

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.1.0
• Component/s: Spark
• Assignee: Chengxiang Li
• Reporter: Chengxiang Li
• Created: 23/Jan/15 07:42
• Updated: 18/Jul/15 19:25
• Resolved: 26/Jan/15 19:06

Description

We only push Spark configuration and RSC configuration to Spark while launch Spark cluster now, for Spark on YARN mode, Spark need extra YARN configuration to launch Spark cluster. Besides this, to support dynamically configuration setting for RSC configuration/YARN configuration, we need to recreate SparkSession while RSC configuration/YARN configuration update as well, as they may influence the Spark cluster deployment as well.

Issue Links

• is depended upon by: HIVE-9211 Research on build mini HoS cluster on YARN for unit test[Spark Branch]
• links to: https://reviews.apache.org/r/30208/