[HIVE-24914] Improve LLAP scheduling by only traversing hosts with capacity

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-1
• Component/s: llap
• Labels: pull-request-available
• Assignee: Panagiotis Garefalakis
• Reporter: Panagiotis Garefalakis
• Created: 19/Mar/21 17:57
• Updated: 17/Nov/22 08:48
• Resolved: 26/Apr/21 17:04

Description

schedulePendingTasks on the LlapTaskScheduler currently goes through all the pending tasks and tries to allocate them based on their Priority – if a priority can not be scheduled completely, we bail out as lower priorities would not be able to get allocations either.

An optimization here could be to only walk through the nodes with capacity (if any) ,and not all available hosts, for scheduling these tasks based on their priority and locality preferences.

Issue Links

• links to: GitHub Pull Request #2108