[HIVE-9099] infer_bucket_sort_convert_join.q and mapjoin_hook.q failed.[Spark Branch]

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.1.0
• Component/s: Spark
• Labels: Spark-M4
• Assignee: Xuefu Zhang
• Reporter: Chengxiang Li
• Created: 15/Dec/14 08:03
• Updated: 29/May/15 02:28
• Resolved: 15/Dec/14 21:25

Description

infer_bucket_sort_convert_join.q and mapjoin_hook.q failed after HIVE-9078 due to same reason.
With the following settings in 2 qfiles:

set hive.auto.convert.join = true;
set hive.mapjoin.localtask.max.memory.usage = 0.0001;
set hive.mapjoin.check.memory.rows = 2;
set hive.auto.convert.join.noconditionaltask = false;

With MR, MapRedTask of join query would failed at first and then driver execute its backup task which would succeed, while we do not have backup task in Spark mode.

Issue Links

• relates to: HIVE-9078 Hive should not submit second SparkTask while previous one has failed.[Spark Branch]
• relates to: HIVE-9103 Support backup task for join related optimization [Spark Branch]