[IMPALA-5120] Consider defaulting to partitioned join when no stats are available.

• Type: Improvement
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 2.9.0
• Fix Version/s: Impala 2.9.0
• Component/s: Frontend
• Target Version: Impala 2.9.0
• Epic Color: ghx-label-6
• Assignee: Thomas Tauber-Marshall
• Reporter: Tim Armstrong
• Created: 27/Mar/17 23:29
• Updated: 26/May/17 22:17
• Resolved: 08/May/17 19:13

Description

We currently default to broadcast join when no stats are available, since the code estimates are both MAX_LONG and in the case of equal costs, broadcast wins. We should consider making partitioned join the default because it will use less memory.

The code is here: https://github.com/apache/incubator-impala/blob/master/fe/src/main/java/org/apache/impala/planner/DistributedPlanner.java#L509

    && (node.getDistributionModeHint() == DistributionMode.BROADCAST
            || (node.getDistributionModeHint() != DistributionMode.PARTITIONED
&& broadcastCost <= partitionCost)))

Issue Links

• relates to: IMPALA-5381 Add query option to control join strategy when tables have no stats