Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-2550

Flaky tests in TestZKUUIDService.java

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.3.0
    • Component/s: tests
    • Labels:
      None

      Description

      1. Test case testMultipleIDGeneration_withMultiThread in TestZKUUIDService uses an ArrayList which is written by two threads simultaneously. This is dangerous and the list must be externally synchronized to prevent race conditions.

      Another problem is that you cannot put items at arbitrary indexes in an ArrayList. For example, if the list is empty, the following code throws ArrayIndexOutOfBoundException:

      List<Boolean> test = new ArrayList<>(10000);
      test.add(22, true);
      

      In an unlucky scheduling event, the following can happen:
      1. The list has a certain number of elements, the value of "size" inside the list is 1000
      2. Thread-1 retrieves the next ID from ZK UUID service, which is 1001
      3. Thread-2 retrieves the next ID from ZK UUID service, which is 1002
      4. Thread-2 runs faster than Thread-1 and tries to call list.add(1002, true) which fails.

      The following error was caught during a test run:

      Tests run: 8, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 53.571 sec <<< FAILURE!
      testMultipleIDGeneration_withMultiThread(org.apache.oozie.service.TestZKUUIDService)  Time elapsed: 0.02 sec  <<< ERROR!
      java.lang.IndexOutOfBoundsException: Index: 89, Size: 89
              at java.util.ArrayList.rangeCheck(ArrayList.java:653)
              at java.util.ArrayList.get(ArrayList.java:429)
              at org.apache.oozie.service.TestZKUUIDService.testMultipleIDGeneration_withMultiThread(TestZKUUIDService.java:134)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:498)
              at junit.framework.TestCase.runTest(TestCase.java:168)
              at junit.framework.TestCase.runBare(TestCase.java:134)
              at junit.framework.TestResult$1.protect(TestResult.java:110)
              at junit.framework.TestResult.runProtected(TestResult.java:128)
              at junit.framework.TestResult.run(TestResult.java:113)
              at junit.framework.TestCase.run(TestCase.java:124)
              at junit.framework.TestSuite.runTest(TestSuite.java:243)
              at junit.framework.TestSuite.run(TestSuite.java:238)
              at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
              at org.junit.runners.Suite.runChild(Suite.java:128)
              at org.junit.runners.Suite.runChild(Suite.java:24)
              at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
              at java.util.concurrent.FutureTask.run(FutureTask.java:266)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
              at java.lang.Thread.run(Thread.java:745)
      

      2. The order in which the tests are executed is random. The problem is that testResetSequence sets the maximum sequence number to 900. Because this value is stored in a static variable inside ZKUUIDService, it affects testIDGeneration and testMultipleIDGeneration if these tests are run after testResetSequence.

        Attachments

        1. OOZIE-2550-005.patch
          4 kB
          Peter Bacsko
        2. OOZIE-2550-006.patch
          4 kB
          Peter Bacsko

          Activity

            People

            • Assignee:
              pbacsko Peter Bacsko
              Reporter:
              pbacsko Peter Bacsko
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: