[AMBARI-20712] Parallel Requests With Intersecting Hosts Don't Block Correctly

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 2.5.1
• Fix Version/s: 2.5.1
• Assignee: Jonathan Hurley
• Reporter: Jonathan Hurley
• Created: 08/Apr/17 19:16
• Updated: 16/Feb/18 13:14
• Resolved: 09/Apr/17 02:26

Description

AMBARI-20646 introduced a regression which can be seen when deploying a cluster via blueprint with Kerberization. The problem is caused by the change which was made to only pull in the most recent stage in progress per request.

There was logic which prevented concurrently executing request stages to be scheduled if a prior request had an un-executed stage with a conflicting host. For example:

• Request 1
  • Stage 1 (IN_PROGRESS)
    • Host 1
    • Host 2
    • Host 3
  • Stage 2 (PENDING)
    • Host 4

• Request 2
  • Stage 1
    • Host 4

In the above scenario, the scheduler was tricked into thinking that Request 2 can run even though Host-4 has a conflict. This is because it was only looking at the stage in progress.

We can't simply look at all stages in progress since this would cause the performance bug to manifest again. So, the solution is to have the database tell us which hosts are blocking from prior requests. This SQL should be very fast and will only run in the specific cases where there are multiple concurrent requests (which is typically at blueprint time).

Issue Links

• links to: Reviewboard