[HIVE-19077] Handle duplicate ptests requests standing in queue at the same time

• Type: Improvement
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-1
• Component/s: Testing Infrastructure
• Assignee: Ádám Szita
• Reporter: Ádám Szita
• Created: 29/Mar/18 13:12
• Updated: 17/Nov/22 08:54
• Resolved: 14/May/18 09:00

Description

I've been keeping on eye on our PreCommit-HIVE-Build job, and what I noticed that sometimes huge queues can build up, that contain jira's more than once. (Yesterday I've seen a queue of 40, having 31 distinct jiras..)

Simple scenario is that I upload a patch, it gets queued for ptest (already long queue), and 3 hours later I will update it, re-upload and re-queue. Now the current ptest infra seems to be smart enough to always deal with the latest patch, so what will happen is that the same patch will be tested 2 times (with ~3 hours) diff, most probably with same result.

I propose we do some deduplication - if ptest starts running the request for Jira X, then it can take a look on the current queue, and see if X is there again. If so, it can skip for now, it will be picked up later anyway.

In practice this means that if you reconsider your patch and update it, your original place in the queue will be gone (like as a penalty for changing it), but overall it saves resources for the whole community.