Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Not A Problem
-
3.3.2
-
None
Description
Our servers do live in Europe/Moscow timezone.
The problem is that oozie coordinator is always 4 hours before nominal time of datasets.
Here is the sample of coordinator workflow:
<coordinator-app name="Url-rating-coordinator" frequency="60" start="${httpCoordStartTime}" end="2113-01-01T00:02Z" timezone="${httpCoordTimeZone}" xmlns="uri:oozie:coordinator:0.3"> <controls> <timeout>2</timeout> <concurrency>1</concurrency> </controls> <datasets> <dataset name="urlRatingInputDataset" frequency="${coord:hours(1)}" initial-instance="${httpCoordStartTime}" timezone="${httpCoordTimeZone}"> <uri-template>${nameNode}/staging/landing/source/protei/http/${YEAR}/${MONTH}/${DAY}/${HOUR}</uri-template> <done-flag></done-flag> </dataset> <dataset name="urlRatingOutputDataset" frequency="${coord:hours(1)}" initial-instance="${httpCoordStartTime}" timezone="${httpCoordTimeZone}"> <uri-template>${nameNode}/masterdata/source/protei/http/archive/${YEAR}/${MONTH}/${DAY}/${HOUR}</uri-template> <done-flag></done-flag> </dataset> </datasets>
Here are the parameters:
httpCoordStartTime=2013-04-21T16:02Z
httpCoordTimeZone=GMT+04:00
Imagine: Right now is 16-22 in Moscow.
The input dataset folder is ready to be processed:
/staging/landing/source/protei/http/${YEAR}/${MONTH}/${DAY}/16
I do submit oozie coordinator with mentioned parameters.
I do expect that Oozie will take input folder
/staging/landing/source/protei/http/${YEAR}/${MONTH}/${DAY}/16
and process it. But it doesn't do it. It says:
2013-04-21 16:22:45,868 INFO org.apache.oozie.command.coord.CoordSubmitXCommand: USER[hdfs] GROUP[-] TOKEN[] APP[Url-rating-coordinator] JOB[0000334-130410234028321-oozie-oozi-C] ACTION[-] ENDED Coordinator Submit jobId=0000334-130410234028321-oozie-oozi-C 2013-04-21 16:22:45,972 WARN org.apache.oozie.command.coord.CoordMaterializeTransitionXCommand: USER[hdfs] GROUP[-] TOKEN[] APP[Url-rating-coordinator] JOB[0000334-130410234028321-oozie-oozi-C] ACTION[-] E1100: Command precondition does not hold before execution, [CoordMaterializeTransitionXCommand for jobId=0000334-130410234028321-oozie-oozi-C job's start time is not reached yet - nothing to materialize], Error Code: E1100
Ok, I try to change params to (roll start time 4 hours back):
httpCoordStartTime=2013-04-21T12:02Z
httpCoordTimeZone=GMT+04:00
Ooize happily starts to process input folder
/staging/landing/source/protei/http/${YEAR}/${MONTH}/${DAY}/12
The same situation, it always for hours before the time I want it to pocess the data.
Is it a bug?