[IMPALA-9015] Use better algorithm for allocating scan ranges to finstances within a daemon in schedule

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 3.4.0
• Component/s: Distributed Exec
• Epic Color: ghx-label-9
• Assignee: Tim Armstrong
• Reporter: Tim Armstrong
• Created: 07/Oct/19 16:03
• Updated: 10/Oct/19 05:05
• Resolved: 10/Oct/19 05:04

Description

Currently the scheduler uses a single-pass algorithm to allocate scan ranges to nodes. It has several deficiencies:

• It doesn't guarantee that the desired number of instances are created. E.g. if mt_dop is 4, there are 3 impalads and 9 scan ranges, it should create 3 instances per impala but doesn't reliably.
• It tends to over-allocate to the first instances it visits.
• The result depends quite a bit on the input order of the scan ranges.