Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-2063

Cron syntax creates duplicate actions

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: trunk, 4.0.0, 4.1.0, 4.0.1
    • Fix Version/s: 4.2.0
    • Component/s: coordinator
    • Labels:
      None

      Description

      If you use cron syntax, you'll get duplicate actions (i.e. actions with the same nominal time) at every throttle interval. For example, if throttle=12 (the default), you'll have a duplicate action every 12 actions.

      Here's my coordinator:

      <coordinator-app name="cron-coord" frequency="*/5 * * * *" start="${start}" end="${end}" timezone="UTC"
                       xmlns="uri:oozie:coordinator:0.2">
        <controls>
          <throttle>3</throttle>
        </controls>
              <action>
              <workflow>
                  <app-path>${workflowAppUri}</app-path>
                  <configuration>
                      <property>
                          <name>jobTracker</name>
                          <value>${jobTracker}</value>
                      </property>
                      <property>
                          <name>nameNode</name>
                          <value>${nameNode}</value>
                      </property>
                      <property>
                          <name>queueName</name>
                          <value>${queueName}</value>
                      </property>
                  </configuration>
              </workflow>
          </action>
      </coordinator-app>
      

      It runs every 5 min on the hour. I also ran a similar coordinator, but with the frequency set to coord:minutes(5). I set the throttle to 3 so it would be easier to see the problem.

      Here are the two screenshots (make sure to either open them directly or widen the page to see the nominal time column):

      As you can see in the cron screenshot, every 3 actions there's a duplicate action. And the next materialization time is the same as the latest materialized action, so it's going to do it again when it materializes more actions. On the minute screenshot, it's behaving correctly. You'll also see that the next materialized action is now 15 minutes ahead of the cron version!

      This has two major problems:

      • You have duplicate actions
      • The coordinator slowly gets further and further behind where it should be

        Attachments

        1. cron.jpg
          244 kB
          Robert Kanter
        2. min.jpg
          231 kB
          Robert Kanter
        3. oozie-2063.patch
          3 kB
          Bowen Zhang
        4. oozie-2063.patch
          3 kB
          Bowen Zhang

          Issue Links

            Activity

              People

              • Assignee:
                bowenzhangusa Bowen Zhang
                Reporter:
                rkanter Robert Kanter
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: