[YARN-4209] RMStateStore FENCED state doesn't work due to updateFencedState called by stateMachine.doTransition

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.7.2
• Fix Version/s: 2.8.0, 2.7.2, 3.0.0-alpha1
• Component/s: resourcemanager
• Hadoop Flags: Reviewed
• Assignee: Zhihai Xu
• Reporter: Zhihai Xu
• Created: 28/Sep/15 07:40
• Updated: 12/May/17 18:00
• Resolved: 07/Oct/15 04:10

Description

RMStateStore FENCED state doesn’t work due to updateFencedState called by stateMachine.doTransition. The reason is
stateMachine.doTransition called from updateFencedState is embedded in stateMachine.doTransition called from public API(removeRMDelegationToken...) or ForwardingEventHandler#handle. So right after the internal state transition from updateFencedState changes the state to FENCED state, the external state transition changes the state back to ACTIVE state. The end result is that RMStateStore is still in ACTIVE state even after notifyStoreOperationFailed is called. The only working case for FENCED state is notifyStoreOperationFailed called from ZKRMStateStore#VerifyActiveStatusThread.
For example: removeRMDelegationToken => handleStoreEvent => enter external stateMachine.doTransition => RemoveRMDTTransition => notifyStoreOperationFailed =>updateFencedState=>handleStoreEvent=> enter internal stateMachine.doTransition => exit internal stateMachine.doTransition change state to FENCED => exit external stateMachine.doTransition change state to ACTIVE.