[CONTINUUM-2514] Prepare Build Queue slow down parallel built start

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.3.6
• Component/s: Parallel Builds
• Labels: backlog-to-cleanup
• Assignee: Unassigned
• Reporter: Philippe Busque
• Created: 29/Apr/10 12:39
• Updated: 20/Nov/12 01:09

Description

When a scheduler or a build group trigger a series of compilation, every projects are enqueued in the Prepare Build Queue. This queue doesn't consider the project group of a project when iterating through it.

As a result, newly added project, but independent, can wait a long time before being launched in a different built queue, as it has to wait for every entries from the previous groups to be processed.

To speed up the launch of parallel builts, the Prepare Build Queue should alternate between Project Group when more than 1 group is present.

Example:
Set up 4 parallel built queues.
Put 100 projects in Group A.
Put 2 projects in Group B.
Launch the built of Group A. All jobs are inserted in the Prepare Build Queue.
Launch the built of Group B. The 2 jobs are inserted in the Prepare Build Queue.

Since the projects of group B are independent to Group A, they should be added to the Built Queue immediately. However, they will only be added once the 100 projects from Group A has been analyzed.

In our case, we have 6 queues and 40 groups. The average busy queue for the first half hour of compile is 4 queues as the prepare queue doesn't push new projects fast enough to fill the queues.