[IMPALA-9020] Move scan range queues to fragment-level and do dynamic scheduling within a node

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Component/s: Backend
• Target Version: Product Backlog
• Epic Color: ghx-label-6
• Assignee: Unassigned
• Reporter: Tim Armstrong
• Created: 08/Oct/19 01:21
• Updated: 14/May/20 20:22
• Resolved: 14/May/20 20:22

Description

It would be nice to dynamically load balance scans between finstances on a node. There's no particular reason why we can't do that - the code is already set up, in HDFS, to pull scan ranges out of a queue.

I think this will be simpler to do once we don't have to support the "old" multithreaded scans, so I'm including this in later cleanup.

Issue Links

• duplicates: IMPALA-9654 Intra-node execution skew increase with mt_dop