[AMBARI-18728] During cluster install, Components get timed out icon while starting

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.5.0
• Assignee: Andrew Onischuk
• Reporter: Vivek Rathod
• Created: 28/Oct/16 14:41
• Updated: 20/Mar/17 22:01
• Resolved: 29/Oct/16 10:39

Description

This was caused by a very tricky race-condition in the way python multiprocessing.thread works resulting in deadlock in ambari_agent.ActionQueue thread.
The problem is the below flow:
If this all these three get executed at the same time (a very rear occasion):
1. Process1 executes queue.get(False)
2. Process2 executes queue.put(largeObjectWhichTakesLongTimeToPut)
3. Someone kills Process2.

This results in deadlock in process1 get. Which is caused by queue locks/semaphores to being released during put of process2.

I have wrote a script test_race_condition.py to emulate this behaviour and indeed could reproduce this and test the fix for it.

Issue Links

• links to: ReviewBoard