[SPARK-30088] Adaptive execution should convert SortMergeJoin to BroadcastJoin when plan generates empty result

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 3.1.0
• Component/s: SQL
• Assignee: Unassigned
• Reporter: EdisonWang
• Created: 01/Dec/19 09:57
• Updated: 16/Mar/20 22:50

Description

Adaptive execution try to  convert SortMergeJoin to BroadcastJoin by checking the `dataSize` metrics of spark plan. However, if the spark plan generates empty result, the `dataSize` metrics is empty due to SQLMetrics's initial value is -1, which could lead to the following check return false

```

private def canBroadcast(plan: LogicalPlan): Boolean =

{ plan.stats.sizeInBytes >= 0 && plan.stats.sizeInBytes <= conf.autoBroadcastJoinThreshold }

```

Issue Links

• links to: GitHub Pull Request #26726