[IMPALA-5547] Improve join cardinality estimation with a more robust FK/PK detection

• Type: Improvement
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 2.5.0, Impala 2.6.0, Impala 2.7.0, Impala 2.8.0, Impala 2.9.0
• Fix Version/s: Impala 2.10.0
• Component/s: Frontend
• Labels: performance planner tpc-ds
• Target Version: Impala 2.10.0
• Epic Color: ghx-label-6
• Assignee: Alexander Behm
• Reporter: Alexander Behm
• Created: 20/Jun/17 22:05
• Updated: 10/Dec/18 17:13
• Resolved: 03/Jul/17 00:32

Description

This JIRA is for tracking improvements to our join-cardinality estimation. In particular, we should improve the handling of many-to-many joins and multi-column joins. It is understood that some cases cannot be reliably detected with our limited metadata and statistics, but we should try our best given those limitations.

Further improving cardinality estimation with new metadata and statistics is tracked elsewhere, e.g.:
IMPALA-2416
IMPALA-3531

Issue Links

• breaks: IMPALA-7564 Conservative FK/PK join type detection with complex equi-join conjuncts
• relates to: IMPALA-7564 Conservative FK/PK join type detection with complex equi-join conjuncts

Sub-Tasks

• Planner incorrectly estimates cardinality for multi column joins: Resolved
• Planner incorrectly estimates cardinality for many to many joins: Resolved