Uploaded image for project: 'Apache Storm'
  1. Apache Storm
  2. STORM-3072

Frequent test failures in storm-sql-core

    Details

      Description

      Seeing test failures in storm-sql-core, sometimes regular test failures, other times JVM crashes.

      testExternalUdf(org.apache.storm.sql.TestStormSql)  Time elapsed: 8.177 sec  <<< ERROR!
      java.lang.RuntimeException: java.lang.RuntimeException: not a leader, current leader is NimbusInfo{host='DESKTOP-AGC8TKM.localdomain', port=6627, isLeader=true}
              at org.apache.storm.daemon.nimbus.Nimbus.submitTopologyWithOpts(Nimbus.java:2952)
              at org.apache.storm.daemon.nimbus.Nimbus.submitTopology(Nimbus.java:2761)
              at org.apache.storm.LocalCluster.submitTopology(LocalCluster.java:378)
              at org.apache.storm.sql.StormSqlLocalClusterImpl.runLocal(StormSqlLocalClusterImpl.java:68)
              at org.apache.storm.sql.TestStormSql.testExternalUdf(TestStormSql.java:214)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:498)
              at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
              at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
              at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
              at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
              at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
              at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
              at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
              at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
              at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
              at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
              at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
              at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
              at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
              at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
              at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
              at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
              at org.junit.runners.Suite.runChild(Suite.java:127)
              at org.junit.runners.Suite.runChild(Suite.java:26)
              at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
              at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
              at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
              at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
              at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
              at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
              at org.apache.maven.surefire.junitcore.JUnitCore.run(JUnitCore.java:55)
              at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:137)
              at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeLazy(JUnitCoreWrapper.java:119)
              at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:87)
              at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:75)
              at org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:161)
              at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:290)
              at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:242)
              at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:121)
      Caused by: java.lang.RuntimeException: not a leader, current leader is NimbusInfo{host='DESKTOP-AGC8TKM.localdomain', port=6627, isLeader=true}
              at org.apache.storm.daemon.nimbus.Nimbus.assertIsLeader(Nimbus.java:1302)
              at org.apache.storm.daemon.nimbus.Nimbus.submitTopologyWithOpts(Nimbus.java:2798)
              ... 41 more
      
      testExternalNestedArrayOutOfBoundAccess(org.apache.storm.sql.TestStormSql)  Time elapsed: 0.598 sec  <<< ERROR!
      KeyNotFoundException(msg:storm-sql-1-1526308521-stormcode.ser)
              at org.apache.storm.blobstore.LocalFsBlobStore.getStoredBlobMeta(LocalFsBlobStore.java:258)
              at org.apache.storm.blobstore.LocalFsBlobStore.getBlobReplication(LocalFsBlobStore.java:422)
              at org.apache.storm.daemon.nimbus.Nimbus.getBlobReplicationCount(Nimbus.java:1443)
              at org.apache.storm.daemon.nimbus.Nimbus.getClusterInfoImpl(Nimbus.java:2593)
              at org.apache.storm.daemon.nimbus.Nimbus.getClusterInfo(Nimbus.java:4183)
              at org.apache.storm.LocalCluster.getClusterInfo(LocalCluster.java:470)
              at org.apache.storm.sql.StormSqlLocalClusterImpl.runLocal(StormSqlLocalClusterImpl.java:71)
              at org.apache.storm.sql.TestStormSql.testExternalNestedArrayOutOfBoundAccess(TestStormSql.java:170)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:498)
              at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
              at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
              at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
              at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
              at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
              at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
              at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
              at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
              at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
              at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
              at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
              at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
              at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
              at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
              at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
              at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
              at org.junit.runners.Suite.runChild(Suite.java:127)
              at org.junit.runners.Suite.runChild(Suite.java:26)
              at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
              at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
              at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
              at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
              at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
              at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
              at org.apache.maven.surefire.junitcore.JUnitCore.run(JUnitCore.java:55)
              at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:137)
              at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeLazy(JUnitCoreWrapper.java:119)
              at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:87)
              at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:75)
              at org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:161)
              at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:290)
              at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:242)
              at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:121)
      

      The cause seems to be similar to https://issues.apache.org/jira/browse/STORM-3065, some of the tests are interfering with each other. Reducing to fork count 1 fixes it.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Srdo Stig Rohde Døssing
                Reporter:
                Srdo Stig Rohde Døssing
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m