[SPARK-23405] The task will hang up when a small table left semi join a big table

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.2.1
• Fix Version/s: 2.4.0
• Component/s: SQL
• Assignee: KaiXinXIaoLei
• Reporter: KaiXinXIaoLei
• Created: 13/Feb/18 09:15
• Updated: 01/Mar/18 16:12
• Resolved: 01/Mar/18 16:10

Description

1. I run a sql: `select ls.cs_order_number from ls left semi join catalog_sales cs on ls.cs_order_number = cs.cs_order_number`, The `ls` table is a small table ,and the number is one. The `catalog_sales` table is a big table,  and the number is 10 billion. The task will be hang up:

 And the sql page is :

Issue Links

• links to: [Github] Pull Request #20670 (KaiXinXiaoLei)