[IMPALA-10395] Evaluate and improve broadcast vs partition costing formula

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: Impala 3.4.0
• Component/s: Frontend
• Epic Color: ghx-label-13
• Assignee: Aman Sinha
• Reporter: Aman Sinha
• Created: 15/Dec/20 00:03
• Updated: 11/Feb/21 23:43

Description

IMPALA-10287 adds tuning knobs for incorporating the join operator's degree of parallelism (dop) and broadcast-to-partition factor in the costing of the build side of a join when comparing broadcast vs partition distribution.

This JIRA is a follow-on to do a performance evaluation by varying the dop and/or the broadcast-to-partition factor and determine what a reasonable correlation function should be.

Issue Links

• is related to: IMPALA-10287 Distribution strategy is sub-optimal for certain queries