[OOZIE-2063] Cron syntax creates duplicate actions

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: trunk, 4.0.0, 4.1.0, 4.0.1
• Fix Version/s: 4.2.0
• Component/s: coordinator
• Assignee: Bowen Zhang
• Reporter: Robert Kanter
• Created: 12/Nov/14 02:30
• Updated: 18/May/15 07:11
• Resolved: 12/Nov/14 23:28

Description

If you use cron syntax, you'll get duplicate actions (i.e. actions with the same nominal time) at every throttle interval. For example, if throttle=12 (the default), you'll have a duplicate action every 12 actions.

Here's my coordinator:

<coordinator-app name="cron-coord" frequency="*/5 * * * *" start="${start}" end="${end}" timezone="UTC"
                 xmlns="uri:oozie:coordinator:0.2">
  <controls>
    <throttle>3</throttle>
  </controls>
        <action>
        <workflow>
            <app-path>${workflowAppUri}</app-path>
            <configuration>
                <property>
                    <name>jobTracker</name>
                    <value>${jobTracker}</value>
                </property>
                <property>
                    <name>nameNode</name>
                    <value>${nameNode}</value>
                </property>
                <property>
                    <name>queueName</name>
                    <value>${queueName}</value>
                </property>
            </configuration>
        </workflow>
    </action>
</coordinator-app>

It runs every 5 min on the hour. I also ran a similar coordinator, but with the frequency set to coord:minutes(5). I set the throttle to 3 so it would be easier to see the problem.

Here are the two screenshots (make sure to either open them directly or widen the page to see the nominal time column):

As you can see in the cron screenshot, every 3 actions there's a duplicate action. And the next materialization time is the same as the latest materialized action, so it's going to do it again when it materializes more actions. On the minute screenshot, it's behaving correctly. You'll also see that the next materialized action is now 15 minutes ahead of the cron version!

This has two major problems:

• You have duplicate actions
• The coordinator slowly gets further and further behind where it should be

Issue Links

• is duplicated by: OOZIE-2018 Coordinator materialization problems with cron syntax