Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-1140

TestLogStreamer.testStreamLog fails when its started within the first 4 seconds after the hour

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.3.2
    • Component/s: tests
    • Labels:
      None

      Description

      If TestLogStreamer.testStreamLog is started within the first 4 seconds after the hour (e.g. 3:00:00 - 3:00:04, 4:00:00 - 4:00:04, etc) then it will fail with this exception:

      junit.framework.AssertionFailedError: expected:<7> but was:<5>
      	at junit.framework.Assert.fail(Assert.java:50)
      	at junit.framework.Assert.failNotEquals(Assert.java:287)
      	at junit.framework.Assert.assertEquals(Assert.java:67)
      	at junit.framework.Assert.assertEquals(Assert.java:199)
      	at junit.framework.Assert.assertEquals(Assert.java:205)
      	at org.apache.oozie.util.TestLogStreamer.testStreamLog(TestLogStreamer.java:144)
      ...
      

      The test creates some log files and sets their modified time based on the starting time of the test; then it verifies that the right files were picked up when trying to stream log files from a 5 second window starting 10 seconds before the starting time of the test. However, for gz files, we use the time from their name instead of the modified file time, and assume they last for an hour; but, they don't store second or minute granularity.

      So, if the test is started within the first 4 seconds, for example at 8:00:03 on 12/20/2012, then the gz file will be named oozie.log-2012-12-20-08.gz and the window we're looking to stream files from is between 7:59:53 and 7:59:58. The start and end times of the gz file would be considered to be 8:00:00 and 9:00:00 respectively, which doesn't overlap with the window (and the test expects the gz file to fall into the window). If the test is started after the first 4 seconds, for example at 8:07:00, then the gz file will be named oozie.log-2012-12-20-08.gz still, but the window would be 8:06:50 to 8:06:55, which overlaps with the gz file start and end times of 8:00:00 and 9:00:00 (so it works as expected).

      One solution is to change the test to use hours instead of seconds for its window.

        Attachments

        1. OOZIE-1140.patch
          6 kB
          Robert Kanter

          Issue Links

            Activity

              People

              • Assignee:
                rkanter Robert Kanter
                Reporter:
                rkanter Robert Kanter
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: