[IMPALA-9317] Improve number of instances estimate for scans in planner

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 4.0
• Component/s: Frontend
• Labels: multithreading
• Epic Link: Impala Multi-threaded query execution
• Epic Color: ghx-label-10
• Assignee: Tim Armstrong
• Reporter: Tim Armstrong
• Created: 22/Jan/20 21:24
• Updated: 01/Apr/20 23:21
• Resolved: 01/Apr/20 23:21

Description

Currently the number of instances (used for various estimates) is calculated as estimated # hosts * mt_dop. This is suboptimal, because in many cases there isn't enough input data to allow creating that many instances per host.

The per-host estimate for scans actually already handles this correctly by capping the hosts estimate at the number of input splits - we should use the same approach for number of instance estimates.