Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-1837

LauncherMainHadoopUtils sensitive to clock skew

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.3.0
    • Component/s: None
    • Labels:
      None
    • Environment:

      Oozie 4.0.0 (CDH5)

      Description

      The method getChildYarnJobs in LauncherMainHadoopUtils can fail with a message like begin > end in range (begin, end): (1399972474014, 1399972473948).

      startTime = Long.parseLong((System.getProperty("oozie.job.launch.time")));
      ....
      gar.setStartRange(startTime, System.currentTimeMillis());
      

      I guess this is happening when the server on which the launch time was set has a different time then the one this task is running on. In our case there was a skew of about 8 seconds which caused all of our jobs that hit this server to fail.

      I understand that skew in clocks is generally not a good idea but I feel that Oozie could be a bit more resilient here or print a better warning maybe?

        Attachments

        1. OOZIE-1837.001.patch
          4 kB
          Robert Kanter

          Activity

            People

            • Assignee:
              rkanter Robert Kanter
              Reporter:
              larsfrancke Lars Francke
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: