Uploaded image for project: 'Geode'
  1. Geode
  2. GEODE-9383

CI Failure: ReconnectDUnitTest.testReconnectWithRequiredRoleRegained FAILED

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.15.0
    • Fix Version/s: None
    • Component/s: tests
    • Labels:
      None

      Description

      The CI's distributed-test-openjdk8 failed with the error below.

       

      10:52:10org.apache.geode.cache30.ReconnectDUnitTest > testReconnectWithRequiredRoleRegained FAILED
      10:52:10    java.lang.AssertionError: Thread did not terminate after 120000 ms: Thread[run invoked on an instance of org.apache.geode.test.dunit.internal.IdentifiableRunnable,5,Async Invocations]
      10:52:10        at org.junit.Assert.fail(Assert.java:89)
      10:52:10        at org.apache.geode.test.dunit.ThreadUtils.join(ThreadUtils.java:147)
      10:52:10        at org.apache.geode.test.dunit.ThreadUtils.join(ThreadUtils.java:110)
      10:52:10        at org.apache.geode.cache30.ReconnectDUnitTest.testReconnectWithRequiredRoleRegained(ReconnectDUnitTest.java:905)
      10:52:10
      10:52:10    java.lang.AssertionError: Suspicious strings were written to the log during this run.
      10:52:10    Fix the strings or use IgnoredException.addIgnoredException to ignore.
      10:52:10    -----------------------------------------------------------------------
      10:52:10    Found suspect string in 'dunit_suspect-vm3.log' at line 837
      10:52:10
      10:52:10    [fatal 2021/06/11 17:50:10.326 GMT <Geode Membership View Creator> tid=253] Membership service failure: Exiting due to possible network partition event due to loss of 1 cache processes: [3c0ddb2b78f9(251)<v5>:41004]
      10:52:10    org.apache.geode.distributed.internal.membership.api.MemberDisconnectedException: Exiting due to possible network partition event due to loss of 1 cache processes: [3c0ddb2b78f9(251)<v5>:41004]
      10:52:10    	at org.apache.geode.distributed.internal.membership.gms.GMSMembership$ManagerImpl.forceDisconnect(GMSMembership.java:1787)
      10:52:10    	at org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave.forceDisconnect(GMSJoinLeave.java:1122)
      10:52:10    	at org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave.access$1300(GMSJoinLeave.java:80)
      10:52:10    	at org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave$ViewCreator.prepareAndSendView(GMSJoinLeave.java:2588)
      10:52:10    	at org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave$ViewCreator.createAndSendView(GMSJoinLeave.java:2565)
      10:52:10    	at org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave$ViewCreator.run(GMSJoinLeave.java:2340)
      10:52:10
      10:52:10    -----------------------------------------------------------------------
      10:52:10    Found suspect string in 'dunit_suspect-vm1.log' at line 692
      10:52:10
      10:52:10    [fatal 2021/06/11 17:50:09.327 GMT <unicast receiver,3c0ddb2b78f9-23916> tid=392] Membership service failure: Member isn't responding to heartbeat requests
      10:52:10    org.apache.geode.distributed.internal.membership.api.MemberDisconnectedException: Member isn't responding to heartbeat requests
      10:52:10    	at org.apache.geode.distributed.internal.membership.gms.GMSMembership$ManagerImpl.forceDisconnect(GMSMembership.java:1787)
      10:52:10    	at org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave.forceDisconnect(GMSJoinLeave.java:1122)
      10:52:10    	at org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave.processRemoveMemberMessage(GMSJoinLeave.java:725)
      10:52:10    	at org.apache.geode.distributed.internal.membership.gms.messenger.JGroupsMessenger$JGroupsReceiver.receive(JGroupsMessenger.java:1367)
      10:52:10    	at org.apache.geode.distributed.internal.membership.gms.messenger.JGroupsMessenger$JGroupsReceiver.receive(JGroupsMessenger.java:1303)
      10:52:10    	at org.jgroups.JChannel.invokeCallback(JChannel.java:816)
      10:52:10    	at org.jgroups.JChannel.up(JChannel.java:741)
      10:52:10    	at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1030)
      10:52:10    	at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
      10:52:10    	at org.jgroups.protocols.FlowControl.up(FlowControl.java:390)
      10:52:10    	at org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1077)
      10:52:10    	at org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:792)
      10:52:10    	at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:433)
      10:52:10    	at org.apache.geode.distributed.internal.membership.gms.messenger.StatRecorder.up(StatRecorder.java:72)
      10:52:10    	at org.apache.geode.distributed.internal.membership.gms.messenger.AddressManager.up(AddressManager.java:70)
      10:52:10    	at org.jgroups.protocols.TP.passMessageUp(TP.java:1658)
      10:52:10    	at org.jgroups.protocols.TP$SingleMessageHandler.run(TP.java:1876)
      10:52:10    	at org.jgroups.util.DirectExecutor.execute(DirectExecutor.java:10)
      10:52:10    	at org.jgroups.protocols.TP.handleSingleMessage(TP.java:1789)
      10:52:10    	at org.jgroups.protocols.TP.receive(TP.java:1714)
      10:52:10    	at org.apache.geode.distributed.internal.membership.gms.messenger.Transport.receive(Transport.java:160)
      10:52:10    	at org.jgroups.protocols.UDP$PacketReceiver.run(UDP.java:701)
      10:52:10    	at java.lang.Thread.run(Thread.java:748)
      10:52:10
      10:52:10    -----------------------------------------------------------------------
      10:52:10    Found suspect string in 'dunit_suspect-vm2.log' at line 555
      10:52:10
      10:52:10    [fatal 2021/06/11 17:50:09.376 GMT <unicast receiver,3c0ddb2b78f9-55561> tid=51] Membership service failure: Membership coordinator 3c0ddb2b78f9(291:locator)<ec><v0>:41001 has declared that a network partition has occurred
      10:52:10    org.apache.geode.distributed.internal.membership.api.MemberDisconnectedException: Membership coordinator 3c0ddb2b78f9(291:locator)<ec><v0>:41001 has declared that a network partition has occurred
      10:52:10    	at org.apache.geode.distributed.internal.membership.gms.GMSMembership$ManagerImpl.forceDisconnect(GMSMembership.java:1787)
      10:52:10    	at org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave.forceDisconnect(GMSJoinLeave.java:1122)
      10:52:10    	at org.apache.geode.distributed.internal.membership.gms.membership.GMSJoinLeave.processNetworkPartitionMessage(GMSJoinLeave.java:1466)
      10:52:10    	at org.apache.geode.distributed.internal.membership.gms.messenger.JGroupsMessenger$JGroupsReceiver.receive(JGroupsMessenger.java:1367)
      10:52:10    	at org.apache.geode.distributed.internal.membership.gms.messenger.JGroupsMessenger$JGroupsReceiver.receive(JGroupsMessenger.java:1303)
      10:52:10    	at org.jgroups.JChannel.invokeCallback(JChannel.java:816)
      10:52:10    	at org.jgroups.JChannel.up(JChannel.java:741)
      10:52:10    	at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1030)
      10:52:10    	at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
      10:52:10    	at org.jgroups.protocols.FlowControl.up(FlowControl.java:390)
      10:52:10    	at org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1077)
      10:52:10    	at org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:792)
      10:52:10    	at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:433)
      10:52:10    	at org.apache.geode.distributed.internal.membership.gms.messenger.StatRecorder.up(StatRecorder.java:72)
      10:52:10    	at org.apache.geode.distributed.internal.membership.gms.messenger.AddressManager.up(AddressManager.java:70)
      10:52:10    	at org.jgroups.protocols.TP.passMessageUp(TP.java:1658)
      10:52:10    	at org.jgroups.protocols.TP$SingleMessageHandler.run(TP.java:1876)
      10:52:10    	at org.jgroups.util.DirectExecutor.execute(DirectExecutor.java:10)
      10:52:10    	at org.jgroups.protocols.TP.handleSingleMessage(TP.java:1789)
      10:52:10    	at org.jgroups.protocols.TP.receive(TP.java:1714)
      10:52:10    	at org.apache.geode.distributed.internal.membership.gms.messenger.Transport.receive(Transport.java:160)
      10:52:10    	at org.jgroups.protocols.UDP$PacketReceiver.run(UDP.java:701)
      10:52:10    	at java.lang.Thread.run(Thread.java:748)
      10:52:10        at org.junit.Assert.fail(Assert.java:89)
      10:52:10        at org.apache.geode.test.dunit.internal.DUnitLauncher.closeAndCheckForSuspects(DUnitLauncher.java:409)
      10:52:10        at org.apache.geode.test.dunit.internal.DUnitLauncher.closeAndCheckForSuspects(DUnitLauncher.java:425)
      10:52:10        at org.apache.geode.test.dunit.internal.JUnit4DistributedTestCase.cleanupAllVms(JUnit4DistributedTestCase.java:550)
      10:52:10        at org.apache.geode.test.dunit.internal.JUnit4DistributedTestCase.doTearDownDistributedTestCase(JUnit4DistributedTestCase.java:497)
      10:52:10        at org.apache.geode.test.dunit.internal.JUnit4DistributedTestCase.tearDownDistributedTestCase(JUnit4DistributedTestCase.java:480)
      10:52:10        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      10:52:10        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      10:52:10        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      10:52:10        at java.lang.reflect.Method.invoke(Method.java:498)
      10:52:10        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
      10:52:10        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
      10:52:10        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
      10:52:10        at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46)
      10:52:10        at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
      10:52:10        at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
      10:52:10        at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
      10:52:10        at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
      10:52:10        at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
      10:52:10        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
      10:52:10        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
      10:52:10        at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
      10:52:10        at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
      10:52:10        at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
      10:52:10        at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
      10:52:10        at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
      10:52:10        at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
      10:52:10        at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
      10:52:10        at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
      10:52:10        at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:110)
      10:52:10        at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
      10:52:10        at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
      10:52:10        at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
      10:52:10        at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
      10:52:10        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      10:52:10        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      10:52:10        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      10:52:10        at java.lang.reflect.Method.invoke(Method.java:498)
      10:52:10        at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
      10:52:10        at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
      10:52:10        at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
      10:52:10        at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
      10:52:10        at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
      10:52:10        at org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:119)
      10:52:10        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      10:52:10        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      10:52:10        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      10:52:10        at java.lang.reflect.Method.invoke(Method.java:498)
      10:52:10        at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
      10:52:10        at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
      10:52:10        at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182)
      10:52:10        at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164)
      10:52:10        at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:414)
      10:52:10        at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
      10:52:10        at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
      10:52:10        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      10:52:10        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      10:52:10        at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
      10:52:10        at java.lang.Thread.run(Thread.java:748) 

       

      =-=-=-=-=-=-=-=-=-=-=-=-=-=-=  Test Results URI =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
      http://files.apachegeode-ci.info/builds/apache-develop-main/1.15.0-build.0315/test-results/distributedTest/1623439337/
      =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
      
      Test report artifacts from this job are available at:
      
      http://files.apachegeode-ci.info/builds/apache-develop-main/1.15.0-build.0315/test-artifacts/1623439337/distributedtestfiles-openjdk8-1.15.0-build.0315.tgz 

       

       

       

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              mhansonp Mark Hanson
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: