Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-2550

Flaky tests in TestZKUUIDService.java

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • None
    • 4.3.0
    • tests
    • None

    Description

      1. Test case testMultipleIDGeneration_withMultiThread in TestZKUUIDService uses an ArrayList which is written by two threads simultaneously. This is dangerous and the list must be externally synchronized to prevent race conditions.

      Another problem is that you cannot put items at arbitrary indexes in an ArrayList. For example, if the list is empty, the following code throws ArrayIndexOutOfBoundException:

      List<Boolean> test = new ArrayList<>(10000);
      test.add(22, true);
      

      In an unlucky scheduling event, the following can happen:
      1. The list has a certain number of elements, the value of "size" inside the list is 1000
      2. Thread-1 retrieves the next ID from ZK UUID service, which is 1001
      3. Thread-2 retrieves the next ID from ZK UUID service, which is 1002
      4. Thread-2 runs faster than Thread-1 and tries to call list.add(1002, true) which fails.

      The following error was caught during a test run:

      Tests run: 8, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 53.571 sec <<< FAILURE!
      testMultipleIDGeneration_withMultiThread(org.apache.oozie.service.TestZKUUIDService)  Time elapsed: 0.02 sec  <<< ERROR!
      java.lang.IndexOutOfBoundsException: Index: 89, Size: 89
              at java.util.ArrayList.rangeCheck(ArrayList.java:653)
              at java.util.ArrayList.get(ArrayList.java:429)
              at org.apache.oozie.service.TestZKUUIDService.testMultipleIDGeneration_withMultiThread(TestZKUUIDService.java:134)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:498)
              at junit.framework.TestCase.runTest(TestCase.java:168)
              at junit.framework.TestCase.runBare(TestCase.java:134)
              at junit.framework.TestResult$1.protect(TestResult.java:110)
              at junit.framework.TestResult.runProtected(TestResult.java:128)
              at junit.framework.TestResult.run(TestResult.java:113)
              at junit.framework.TestCase.run(TestCase.java:124)
              at junit.framework.TestSuite.runTest(TestSuite.java:243)
              at junit.framework.TestSuite.run(TestSuite.java:238)
              at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
              at org.junit.runners.Suite.runChild(Suite.java:128)
              at org.junit.runners.Suite.runChild(Suite.java:24)
              at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
              at java.util.concurrent.FutureTask.run(FutureTask.java:266)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
              at java.lang.Thread.run(Thread.java:745)
      

      2. The order in which the tests are executed is random. The problem is that testResetSequence sets the maximum sequence number to 900. Because this value is stored in a static variable inside ZKUUIDService, it affects testIDGeneration and testMultipleIDGeneration if these tests are run after testResetSequence.

      Attachments

        1. OOZIE-2550-005.patch
          4 kB
          Peter Bacsko
        2. OOZIE-2550-006.patch
          4 kB
          Peter Bacsko

        Activity

          People

            pbacsko Peter Bacsko
            pbacsko Peter Bacsko
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: