Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-2724

coord:current resolves monthly/yearly dependencies incorrectly

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 4.2.0
    • 4.3.0
    • None
    • None

    Description

      We calculate the difference between two dates to get the instance count. Consider a case where, initial instance = Thu Dec 31 16:00:00 PST 2009 and effective date (nominal time) = Sun Oct 30 17:55:00 PDT 2016. Frequency is monthly. So the instance count would be simply number of months between these two dates. The number of months between are 81 (inclusively). But following code returns 83. A later part of code decreases (possibly some offset deletion logic) this by 1, making it 82.

      Calendar org.apache.oozie.coord.CoordELFunctions.getCurrentInstance(Date effectiveTime, int[] instanceCount, ELEvaluator eval)
      
      ...
      ...
                  case END_OF_MONTH:
                      instanceCount[0] = (int) ((effectiveTime.getTime() - datasetInitialInstance.getTime()) / MONTH_MSEC);
                      break;
      ....
      

      later part of code which is reducing the value by 1:

              if (instanceCount[0] > 2) {
                  instanceCount[0] = (instanceCount[0] / dsFreq);
                  current.add(dsTimeUnit.getCalendarUnit(), instanceCount[0] * dsFreq);
              } else {
                  instanceCount[0] = 0;
              }
              while (!current.getTime().after(effectiveTime)) {
                  current.add(dsTimeUnit.getCalendarUnit(), dsFreq);
                  instanceCount[0]++;
              }
              current.add(dsTimeUnit.getCalendarUnit(), -dsFreq);
              instanceCount[0]--;
              return current;
      

      This happens because there we consider only 30 number of days in a month while calculating the milliseconds in a month. It will also affect yearly jobs because leap year has 366 days.

          public static final long MONTH_MSEC = 30 * DAY_MSEC;
          public static final long YEAR_MSEC = 365 * DAY_MSEC;
      

      Attachments

        1. OOZIE-2724-1.patch
          4 kB
          Satish Saley
        2. OOZIE-2724-2.patch
          4 kB
          Satish Saley

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            satishsaley Satish Saley
            satishsaley Satish Saley
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment