[HIVE-17040] Join elimination in the presence of FK relationship

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 4.0.0-alpha-1
• Component/s: Logical Optimizer
• Assignee: Jesús Camacho Rodríguez
• Reporter: Jesús Camacho Rodríguez
• Created: 05/Jul/17 12:50
• Updated: 27/Feb/24 22:24
• Resolved: 17/Sep/18 15:17

Description

If the PK/UK table is not filtered, we can safely remove the join.

A simple example:

SELECT c_current_cdemo_sk
FROM customer, customer_address
ON c_current_addr_sk = ca_address_sk;

As a Calcite rule, we could implement this rewriting by 1) matching a Project on top of a Join operator, 2) checking that only columns from the FK are used in the Project, 3) checking that the join condition matches the FK - PK/UK relationship, 4) pulling all the predicates from the PK/UK side and checking that the input is not filtered, and 5) removing the join, possibly adding a IS NOT NULL condition on the join column from the FK side.

If the PK/UK table is filtered, we should still transform the Join into a SemiJoin operator.

Issue Links

• duplicates: HIVE-20333 CBO: Join removal based on PK-FK declared constraints
• incorporates: HIVE-17044 Transform LEFT/RIGHT OUTER into INNER join in the presence of FK relationship
• is required by: HIVE-27008 Fix multi_in_clause.q and current_date_timestamp tests in branch-3
• supercedes: HIVE-20501 Vectorization: Closed range fast-path for Fast Long hashset
• links to: RB link