Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-1336

Can't force Oozie to work in local time

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Not A Problem
    • 3.3.2
    • None
    • coordinator

    Description

      Our servers do live in Europe/Moscow timezone.
      The problem is that oozie coordinator is always 4 hours before nominal time of datasets.
      Here is the sample of coordinator workflow:

       
      <coordinator-app name="Url-rating-coordinator" frequency="60"
                       start="${httpCoordStartTime}" end="2113-01-01T00:02Z" timezone="${httpCoordTimeZone}"  xmlns="uri:oozie:coordinator:0.3">
          <controls>
              <timeout>2</timeout>
              <concurrency>1</concurrency>
          </controls>
      
          <datasets>
              <dataset name="urlRatingInputDataset" frequency="${coord:hours(1)}" initial-instance="${httpCoordStartTime}" timezone="${httpCoordTimeZone}">
                  <uri-template>${nameNode}/staging/landing/source/protei/http/${YEAR}/${MONTH}/${DAY}/${HOUR}</uri-template>
                  <done-flag></done-flag>
              </dataset>
              <dataset name="urlRatingOutputDataset" frequency="${coord:hours(1)}" initial-instance="${httpCoordStartTime}" timezone="${httpCoordTimeZone}">
                  <uri-template>${nameNode}/masterdata/source/protei/http/archive/${YEAR}/${MONTH}/${DAY}/${HOUR}</uri-template>
                  <done-flag></done-flag>
              </dataset>
          </datasets>
      

       
      Here are the parameters:
      httpCoordStartTime=2013-04-21T16:02Z
      httpCoordTimeZone=GMT+04:00

      Imagine: Right now is 16-22 in Moscow.
      The input dataset folder is ready to be processed:

       
      /staging/landing/source/protei/http/${YEAR}/${MONTH}/${DAY}/16
      

      I do submit oozie coordinator with mentioned parameters.
      I do expect that Oozie will take input folder

       
      /staging/landing/source/protei/http/${YEAR}/${MONTH}/${DAY}/16 
      

      and process it. But it doesn't do it. It says:

       
      2013-04-21 16:22:45,868 INFO org.apache.oozie.command.coord.CoordSubmitXCommand: USER[hdfs] GROUP[-] TOKEN[] APP[Url-rating-coordinator] JOB[0000334-130410234028321-oozie-oozi-C] ACTION[-] ENDED Coordinator Submit jobId=0000334-130410234028321-oozie-oozi-C
      2013-04-21 16:22:45,972 WARN org.apache.oozie.command.coord.CoordMaterializeTransitionXCommand: USER[hdfs] GROUP[-] TOKEN[] APP[Url-rating-coordinator] JOB[0000334-130410234028321-oozie-oozi-C] ACTION[-] E1100: Command precondition does not hold before execution, [CoordMaterializeTransitionXCommand for jobId=0000334-130410234028321-oozie-oozi-C job's start time is not reached yet - nothing to materialize], Error Code: E1100
      

       
      Ok, I try to change params to (roll start time 4 hours back):
      httpCoordStartTime=2013-04-21T12:02Z
      httpCoordTimeZone=GMT+04:00

      Ooize happily starts to process input folder

       
      /staging/landing/source/protei/http/${YEAR}/${MONTH}/${DAY}/12
      

       
      The same situation, it always for hours before the time I want it to pocess the data.

      Is it a bug?

      Attachments

        Activity

          People

            Unassigned Unassigned
            serega_sheypak Sergey
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: