[HIVE-9078] Hive should not submit second SparkTask while previous one has failed.[Spark Branch]

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.1.0
• Component/s: Spark
• Labels: Spark-M4
• Assignee: Chengxiang Li
• Reporter: Chengxiang Li
• Created: 11/Dec/14 06:47
• Updated: 29/May/15 02:31
• Resolved: 15/Dec/14 17:34

Description

hive> select n_name, c_name from nation, customer where nation.n_nationkey = customer.c_nationkey limit 10;
Query ID = root_20141211135050_51e5ae15-49a3-4a46-826f-e27ee314ccb2
Total jobs = 2
Launching Job 1 out of 2
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapreduce.job.reduces=<number>
Status: Failed
Launching Job 2 out of 2
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapreduce.job.reduces=<number>
Status: Failed
OK
Time taken: 68.53 seconds

2 issue in the above CLI output.

1. For a query which would be translated into multi SparkTask, is previous SparkTask failed, Hive should failed right away, the following SparkTask should not be submitted any more.
2. Print failed info in Hive console while query failed.

The correct CLI output while query failed:

hive> select n_name, c_name from nation, customer where nation.n_nationkey = customer.c_nationkey limit 10;
Query ID = root_20141211142929_ddb7f205-8422-44b4-96bd-96a1c9291895
Total jobs = 2
Launching Job 1 out of 2
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapreduce.job.reduces=<number>
Status: Failed
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.spark.SparkTask

Issue Links

• is related to: HIVE-9099 infer_bucket_sort_convert_join.q and mapjoin_hook.q failed.[Spark Branch]
• links to: https://reviews.apache.org/r/28934/