[SPARK-15302] Implement FK/PK "rely novalidate" constraints for better CBO

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 2.0.0
• Component/s: Optimizer, Spark Core
• Labels: bulk-closed
• Assignee: Unassigned
• Reporter: Ruslan Dautkhanov
• Created: 12/May/16 18:41
• Updated: 21/May/19 04:11
• Resolved: 21/May/19 04:11

Description

Oracle has "RELY NOVALIDATE" option for constraints.. Could be easier for Hive to start with something like that for PK/FK constraints. So CBO has more information for optimizations. It does not have to actually check if that constraint is relationship is true; it can just "rely" on that constraint.

RELY clause:
https://docs.oracle.com/database/121/SQLRF/clauses002.htm#SQLRF52223

"Overview of Constraint States":
https://docs.oracle.com/database/121/DWHSG/schemas.htm#DWHSG9053

• Enforcement
• Validation
• Belief

So FK/PK with "rely novalidate" will have Enforcement&Validate disabled but Belief = RELY as it is possible to do in Oracle and now in Hive (HIVE-13076).

It opens a lot of ways to do additional ways to optimize execution plans.
As explained in Tom Kyte's "Metadata matters"
http://www.peoug.org/wp-content/uploads/2009/12/MetadataMatters_PEOUG_Day2009_TKyte.pdf
pp.30 - "Tell us how the tables relate and we can remove them from the plan...".
pp.35 - "Tell us how the tables relate and we have more access paths available...".

Issue Links

• relates to: SPARK-19842 Informational Referential Integrity Constraints Support in Spark
• relates to: HIVE-13076 Implement FK/PK "rely novalidate" constraints for better CBO