[HIVE-22238] PK/FK selectivity estimation underscales estimations

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-1
• Component/s: Statistics
• Labels: pull-request-available
• Assignee: Zoltan Haindrich
• Reporter: Zoltan Haindrich
• Created: 24/Sep/19 14:24
• Updated: 27/Feb/24 22:24
• Resolved: 08/Nov/19 14:16

Description

at this point the parent operators rownum is scaled according to pkfkselectivity

however pkfkselectivity is computed on a whole subtree.

Scaling it by that amount will count in estimation already used when parentstats was calculated...so depending on the number of upstream joins - this may lead to severe underestimations

what happened was:

• optimization was able to push the filter to the other side of the join
• as a result the incoming data was already filtered
• scaling down by the PK selectiviy - was actually already there...but a new "scaling" happened

Issue Links

• links to: GitHub Pull Request #831