[HIVE-10476] Hive query should fail when it fails to initialize a session in SetSparkReducerParallelism [Spark Branch]

• Type: Sub-task
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: spark-branch
• Fix Version/s: 1.3.0, 2.0.0
• Component/s: Spark
• Assignee: Chao Sun
• Reporter: Chao Sun
• Created: 24/Apr/15 01:15
• Updated: 17/Jan/23 05:46
• Resolved: 28/Apr/15 16:40

Description

Currently, for a Hive query HoS need to get a session
a session twice, once in SparkSetReducerParallelism, and another when submitting the actual job.
The issue is that sometimes there's problem when launching a Yarn application (e.g., don't have permission), then user will have to wait for two timeouts, because both session initializations will fail. This turned out to happen frequently.

This JIRA proposes to fail the query in SparkSetReducerParallelism, when it cannot initialize the session.

Issue Links

• duplicates: HIVE-12649 Hive on Spark will resubmitted application when not enough resouces to launch yarn application master