[HIVE-23210] Fix shortestjobcomparator when jobs submitted have 1 task their vertices

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-1
• Component/s: llap
• Labels: pull-request-available
• Assignee: Panagiotis Garefalakis
• Reporter: Rajesh Balamohan
• Created: 15/Apr/20 07:35
• Updated: 17/Nov/22 08:50
• Resolved: 21/Apr/20 09:07

Description

In latency sensitive queries, lots of jobs can have vertices with 1 task. Currently shortestjobcomparator does not work correctly and returns tasks in random order.

https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/comparator/ShortestJobFirstComparator.java#L51

This causes delay in the job runtime. I will attach a simple test case shortly.

Issue Links

• is related to: HIVE-23443 LLAP speculative task pre-emption seems to be not working
• relates to: HIVE-23238 FIX PreemptionQueueComparator edge cases
• links to: GitHub Pull Request #984