[OOZIE-614] Oozie does not behave as expected when using coordinator execution order LAST_ONLY

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Assignee: Ryota Egashira
• Reporter: Craig Peters
• Created: 23/Nov/11 19:15
• Updated: 17/Apr/14 01:17
• Resolved: 17/Apr/14 01:17

Description

After executing the last coordinator action on a queue for a job, the prior coordinator actions will still be executed if a later coordinator action is not created. The behavior expected based upon the documentation for Coordinator is that the older coordinator actions are discarded. See: http://yahoo.github.com/oozie/releases/3.1.0/CoordinatorFunctionalSpec.html#a6.3._Synchronous_Coordinator_Application_Definition

When using the LAST_ONLY execution order the coordinator job needs to somehow discard the older coordinator actions as the documentation describes. This may result the desired behavior for many users when in steady state operation.

Using a simple approach of invalidating or killing prior coordinator actions there are likely to be cases when this functionality won't behave as expected because at any given time the latest coordinator on the queue in the READY state will not be the latest potential. This is because the newer actions haven't been created yet due to the batch creation of coordinator actions oldest first. Some discussion of the best way to address this challenge is warranted I believe.

Another consideration is that the user will not be able to easily discriminate between coordinator actions that were discarded (or skipped) due to the execution order and those that were killed for any other reason. Perhaps it makes sense to introduce a new state? It could be DISCARDED to match the current documentation or perhaps SKIPPED.

Issue Links

• duplicates: OOZIE-1319 "LAST_ONLY" in execution control for coordinator job still runs all the actions