[YARN-212] NM state machine ignores an APPLICATION_CONTAINER_FINISHED event when it shouldn't

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.23.4, 2.0.1-alpha
• Fix Version/s: 2.0.3-alpha, 0.23.5
• Component/s: nodemanager
• Assignee: Nathan Roberts
• Reporter: Nathan Roberts
• Created: 12/Nov/12 20:45
• Updated: 06/Jun/13 14:21
• Resolved: 13/Nov/12 16:08

Description

The NM state machines can make the following two invalid state transitions when a speculative attempt is killed shortly after it gets started. When this happens the NM keeps the log aggregation context open for this application and therefore chews up FDs and leases on the NN, eventually running the NN out of FDs and bringing down the entire cluster.

2012-11-07 05:36:33,774 [AsyncDispatcher event handler] WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application: Can't handle this event at current state
org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event: APPLICATION_CONTAINER_FINISHED at INITING

2012-11-07 05:36:33,775 [AsyncDispatcher event handler] WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: Can't handle this event at current state: Current: [DONE], eventType: [INIT_CONTAINER]
org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event: INIT_CONTAINER at DONE

Issue Links

• is duplicated by: YARN-346 InvalidStateTransitonException: Invalid event: INIT_CONTAINER at DONE for ContainerImpl in Node Manager