Uploaded image for project: 'Apache Storm'
  1. Apache Storm
  2. STORM-915 Track and resolve any further random tests failure
  3. STORM-916

test-noop-authorization-w-simple-transport in nimbus-auth-test could throws SocketTimeoutException

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Minor
    • Resolution: Cannot Reproduce
    • None
    • None
    • storm-core
    • TravisCI

    Description

      https://travis-ci.org/apache/storm/jobs/68401296
      Relevant PR is https://github.com/apache/storm/pull/593

      One of two build is failed for below reason.

      Checking ./storm-core/target/test-reports/backtype.storm.security.auth.nimbus-auth-test.xml
      --------------------------------------------------
      classname: backtype.storm.security.auth.nimbus-auth-test / testname: test-noop-authorization-w-simple-transport
      Uncaught exception, not in assertion.
      expected: nil
      actual: java.lang.RuntimeException: org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException
      at backtype.storm.security.auth.TBackoffConnect.retryNext (TBackoffConnect.java:59)
      backtype.storm.security.auth.TBackoffConnect.doConnectWithRetry (TBackoffConnect.java:51)
      backtype.storm.security.auth.ThriftClient.reconnect (ThriftClient.java:103)
      backtype.storm.security.auth.ThriftClient.<init> (ThriftClient.java:72)
      backtype.storm.utils.NimbusClient.<init> (NimbusClient.java:64)
      backtype.storm.security.auth.nimbus_auth_test/fn (nimbus_auth_test.clj:77)
      clojure.test$test_var$fn__7187.invoke (test.clj:704)
      clojure.test$test_var.invoke (test.clj:704)
      clojure.test$test_vars$fn_7209$fn_7214.invoke (test.clj:722)
      clojure.test$default_fixture.invoke (test.clj:674)
      clojure.test$test_vars$fn__7209.invoke (test.clj:722)
      clojure.test$default_fixture.invoke (test.clj:674)
      clojure.test$test_vars.invoke (test.clj:718)
      clojure.test$test_all_vars.invoke (test.clj:728)
      clojure.test$test_ns.invoke (test.clj:747)
      clojure.core$map$fn__4245.invoke (core.clj:2559)
      clojure.lang.LazySeq.sval (LazySeq.java:40)
      clojure.lang.LazySeq.seq (LazySeq.java:49)
      clojure.lang.Cons.next (Cons.java:39)
      clojure.lang.RT.boundedLength (RT.java:1654)
      clojure.lang.RestFn.applyTo (RestFn.java:130)
      clojure.core$apply.invoke (core.clj:626)
      clojure.test$run_tests.doInvoke (test.clj:762)
      clojure.lang.RestFn.invoke (RestFn.java:408)
      org.apache.storm.testrunner$eval8792$iter_87938797$fn8798$fn8799$fn_8800.invoke (test_runner.clj:107)
      org.apache.storm.testrunner$eval8792$iter_87938797$fn8798$fn_8799.invoke (test_runner.clj:53)
      org.apache.storm.testrunner$eval8792$iter_87938797$fn_8798.invoke (test_runner.clj:52)
      clojure.lang.LazySeq.sval (LazySeq.java:40)
      clojure.lang.LazySeq.seq (LazySeq.java:49)
      clojure.lang.RT.seq (RT.java:484)
      clojure.core$seq.invoke (core.clj:133)
      clojure.core$dorun.invoke (core.clj:2855)
      org.apache.storm.testrunner$eval8792.invoke (test_runner.clj:52)
      clojure.lang.Compiler.eval (Compiler.java:6703)
      clojure.lang.Compiler.load (Compiler.java:7130)
      clojure.lang.Compiler.loadFile (Compiler.java:7086)
      clojure.main$load_script.invoke (main.clj:274)
      clojure.main$script_opt.invoke (main.clj:336)
      clojure.main$main.doInvoke (main.clj:420)
      clojure.lang.RestFn.invoke (RestFn.java:421)
      clojure.lang.Var.invoke (Var.java:383)
      clojure.lang.AFn.applyToHelper (AFn.java:156)
      clojure.lang.Var.applyTo (Var.java:700)
      clojure.main.main (main.java:37)
      Caused by: org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException
      at org.apache.thrift.transport.TSocket.open (TSocket.java:187)
      org.apache.thrift.transport.TFramedTransport.open (TFramedTransport.java:81)
      backtype.storm.security.auth.SimpleTransportPlugin.connect (SimpleTransportPlugin.java:103)
      backtype.storm.security.auth.TBackoffConnect.doConnectWithRetry (TBackoffConnect.java:48)
      backtype.storm.security.auth.ThriftClient.reconnect (ThriftClient.java:103)
      backtype.storm.security.auth.ThriftClient.<init> (ThriftClient.java:72)
      backtype.storm.utils.NimbusClient.<init> (NimbusClient.java:64)
      backtype.storm.security.auth.nimbus_auth_test/fn (nimbus_auth_test.clj:77)
      clojure.test$test_var$fn__7187.invoke (test.clj:704)
      clojure.test$test_var.invoke (test.clj:704)
      clojure.test$test_vars$fn_7209$fn_7214.invoke (test.clj:722)
      clojure.test$default_fixture.invoke (test.clj:674)
      clojure.test$test_vars$fn__7209.invoke (test.clj:722)
      clojure.test$default_fixture.invoke (test.clj:674)
      clojure.test$test_vars.invoke (test.clj:718)
      clojure.test$test_all_vars.invoke (test.clj:728)
      clojure.test$test_ns.invoke (test.clj:747)
      clojure.core$map$fn__4245.invoke (core.clj:2559)
      clojure.lang.LazySeq.sval (LazySeq.java:40)
      clojure.lang.LazySeq.seq (LazySeq.java:49)
      clojure.lang.Cons.next (Cons.java:39)
      clojure.lang.RT.boundedLength (RT.java:1654)
      clojure.lang.RestFn.applyTo (RestFn.java:130)
      clojure.core$apply.invoke (core.clj:626)
      clojure.test$run_tests.doInvoke (test.clj:762)
      clojure.lang.RestFn.invoke (RestFn.java:408)
      org.apache.storm.testrunner$eval8792$iter_87938797$fn8798$fn8799$fn_8800.invoke (test_runner.clj:107)
      org.apache.storm.testrunner$eval8792$iter_87938797$fn8798$fn_8799.invoke (test_runner.clj:53)
      org.apache.storm.testrunner$eval8792$iter_87938797$fn_8798.invoke (test_runner.clj:52)
      clojure.lang.LazySeq.sval (LazySeq.java:40)
      clojure.lang.LazySeq.seq (LazySeq.java:49)
      clojure.lang.RT.seq (RT.java:484)
      clojure.core$seq.invoke (core.clj:133)
      clojure.core$dorun.invoke (core.clj:2855)
      org.apache.storm.testrunner$eval8792.invoke (test_runner.clj:52)
      clojure.lang.Compiler.eval (Compiler.java:6703)
      clojure.lang.Compiler.load (Compiler.java:7130)
      clojure.lang.Compiler.loadFile (Compiler.java:7086)
      clojure.main$load_script.invoke (main.clj:274)
      clojure.main$script_opt.invoke (main.clj:336)
      clojure.main$main.doInvoke (main.clj:420)
      clojure.lang.RestFn.invoke (RestFn.java:421)
      clojure.lang.Var.invoke (Var.java:383)
      clojure.lang.AFn.applyToHelper (AFn.java:156)
      clojure.lang.Var.applyTo (Var.java:700)
      clojure.main.main (main.java:37)
      Caused by: java.net.SocketTimeoutException: null
      at java.net.SocksSocketImpl.remainingMillis (SocksSocketImpl.java:111)
      java.net.SocksSocketImpl.connect (SocksSocketImpl.java:392)
      java.net.Socket.connect (Socket.java:589)
      org.apache.thrift.transport.TSocket.open (TSocket.java:182)
      org.apache.thrift.transport.TFramedTransport.open (TFramedTransport.java:81)
      backtype.storm.security.auth.SimpleTransportPlugin.connect (SimpleTransportPlugin.java:103)
      backtype.storm.security.auth.TBackoffConnect.doConnectWithRetry (TBackoffConnect.java:48)
      backtype.storm.security.auth.ThriftClient.reconnect (ThriftClient.java:103)
      backtype.storm.security.auth.ThriftClient.<init> (ThriftClient.java:72)
      backtype.storm.utils.NimbusClient.<init> (NimbusClient.java:64)
      backtype.storm.security.auth.nimbus_auth_test/fn (nimbus_auth_test.clj:77)
      clojure.test$test_var$fn__7187.invoke (test.clj:704)
      clojure.test$test_var.invoke (test.clj:704)
      clojure.test$test_vars$fn_7209$fn_7214.invoke (test.clj:722)
      clojure.test$default_fixture.invoke (test.clj:674)
      clojure.test$test_vars$fn__7209.invoke (test.clj:722)
      clojure.test$default_fixture.invoke (test.clj:674)
      clojure.test$test_vars.invoke (test.clj:718)
      clojure.test$test_all_vars.invoke (test.clj:728)
      clojure.test$test_ns.invoke (test.clj:747)
      clojure.core$map$fn__4245.invoke (core.clj:2559)
      clojure.lang.LazySeq.sval (LazySeq.java:40)
      clojure.lang.LazySeq.seq (LazySeq.java:49)
      clojure.lang.Cons.next (Cons.java:39)
      clojure.lang.RT.boundedLength (RT.java:1654)
      clojure.lang.RestFn.applyTo (RestFn.java:130)
      clojure.core$apply.invoke (core.clj:626)
      clojure.test$run_tests.doInvoke (test.clj:762)
      clojure.lang.RestFn.invoke (RestFn.java:408)
      org.apache.storm.testrunner$eval8792$iter_87938797$fn8798$fn8799$fn_8800.invoke (test_runner.clj:107)
      org.apache.storm.testrunner$eval8792$iter_87938797$fn8798$fn_8799.invoke (test_runner.clj:53)
      org.apache.storm.testrunner$eval8792$iter_87938797$fn_8798.invoke (test_runner.clj:52)
      clojure.lang.LazySeq.sval (LazySeq.java:40)
      clojure.lang.LazySeq.seq (LazySeq.java:49)
      clojure.lang.RT.seq (RT.java:484)
      clojure.core$seq.invoke (core.clj:133)
      clojure.core$dorun.invoke (core.clj:2855)
      org.apache.storm.testrunner$eval8792.invoke (test_runner.clj:52)
      clojure.lang.Compiler.eval (Compiler.java:6703)
      clojure.lang.Compiler.load (Compiler.java:7130)
      clojure.lang.Compiler.loadFile (Compiler.java:7086)
      clojure.main$load_script.invoke (main.clj:274)
      clojure.main$script_opt.invoke (main.clj:336)
      clojure.main$main.doInvoke (main.clj:420)
      clojure.lang.RestFn.invoke (RestFn.java:421)
      clojure.lang.Var.invoke (Var.java:383)
      clojure.lang.AFn.applyToHelper (AFn.java:156)
      clojure.lang.Var.applyTo (Var.java:700)
      clojure.main.main (main.java:37)

      at: test_runner.clj:105

      -------------------- system-out --------------------

93518 [main] INFO  b.s.zookeeper - Starting inprocess zookeeper at port 2000 and dir /tmp/2e907bc8-728a-4264-9633-8286457416f9
93520 [main] INFO  b.s.u.Utils - Using defaults.yaml from resources
93527 [main] INFO  b.s.d.nimbus - Starting Nimbus with conf {"topology.builtin.metrics.bucket.size.secs" 60, "nimbus.childopts" "-Xmx1024m", "ui.filter.params" nil, "storm.cluster.mode" "local", "storm.messaging.netty.client_worker_threads" 1, "supervisor.run.worker.as.user" false, "topology.max.task.parallelism" nil, "zmq.threads" 1, "storm.group.mapping.service" "backtype.storm.security.auth.ShellBasedGroupsMapping", "transactional.zookeeper.root" "/transactional", "topology.sleep.spout.wait.strategy.time.ms" 1, "drpc.invocations.port" 3773, "topology.multilang.serializer" "backtype.storm.multilang.JsonSerializer", "storm.messaging.netty.server_worker_threads" 1, "topology.max.error.report.per.interval" 5, "storm.thrift.transport" "backtype.storm.security.auth.SimpleTransportPlugin", "zmq.hwm" 0, "storm.principal.tolocal" "backtype.storm.security.auth.DefaultPrincipalToLocal", "supervisor.worker.shutdown.sleep.secs" 1, "storm.zookeeper.retry.times" 5, "ui.actions.enabled" true, "zmq.linger.millis" 0, "supervisor.enable" true, "topology.stats.sample.rate" 0.05, "storm.messaging.netty.min_wait_ms" 100, "storm.zookeeper.port" 2000, "supervisor.heartbeat.frequency.secs" 5, "topology.enable.message.timeouts" false, "drpc.worker.threads" 64, "drpc.queue.size" 128, "drpc.https.keystore.password" "", "logviewer.port" 8000, "nimbus.reassign" true, "topology.executor.send.buffer.size" 1024, "topology.spout.wait.strategy" "backtype.storm.spout.SleepSpoutWaitStrategy", "ui.host" "0.0.0.0", "storm.nimbus.retry.interval.millis" 2000, "nimbus.inbox.jar.expiration.secs" 3600, "dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.acker.executors" nil, "topology.fall.back.on.java.serialization" true, "storm.zookeeper.servers" ["localhost"], "nimbus.thrift.threads" 64, "logviewer.cleanup.age.mins" 10080, "topology.worker.childopts" nil, "topology.classpath" nil, "supervisor.monitor.frequency.secs" 3, "nimbus.credential.renewers.freq.secs" 600, "topology.skip.missing.kryo.registrations" true, "drpc.authorizer.acl.filename" "drpc-auth-acl.yaml", "storm.group.mapping.service.cache.duration.secs" 120, "topology.testing.always.try.serialize" false, "nimbus.monitor.freq.secs" 10, "supervisor.supervisors" [], "topology.tasks" nil, "topology.bolts.outgoing.overflow.buffer.enable" false, "storm.messaging.netty.socket.backlog" 500, "topology.workers" 1, "storm.local.dir" "/tmp/f9d407e6-abcf-4432-9ddb-052e0391d1e4", "worker.childopts" "-Xmx768m", "storm.auth.simple-white-list.users" [], "topology.message.timeout.secs" 30, "topology.state.synchronization.timeout.secs" 60, "topology.tuple.serializer" "backtype.storm.serialization.types.ListDelegateSerializer", "supervisor.supervisors.commands" [], "logviewer.childopts" "-Xmx128m", "topology.environment" nil, "topology.debug" false, "storm.messaging.netty.max_retries" 300, "ui.childopts" "-Xmx768m", "storm.zookeeper.session.timeout" 20000, "drpc.childopts" "-Xmx768m", "drpc.http.creds.plugin" "backtype.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.connection.timeout" 15000, "storm.zookeeper.auth.user" nil, "storm.meta.serialization.delegate" "backtype.storm.serialization.GzipThriftSerializationDelegate", "topology.max.spout.pending" nil, "nimbus.supervisor.timeout.secs" 60, "nimbus.task.timeout.secs" 30, "drpc.port" 3772, "storm.zookeeper.retry.intervalceiling.millis" 30000, "nimbus.thrift.port" 60040, "storm.auth.simple-acl.admins" [], "storm.nimbus.retry.times" 5, "supervisor.worker.start.timeout.secs" 120, "storm.zookeeper.retry.interval" 1000, "logs.users" nil, "transactional.zookeeper.port" nil, "drpc.max_buffer_size" 1048576, "task.credentials.poll.secs" 30, "drpc.https.keystore.type" "JKS", "topology.worker.receiver.thread.count" 1, "supervisor.slots.ports" [6700 6701 6702 6703], "topology.transfer.buffer.size" 1024, "topology.worker.shared.thread.pool.size" 4, "drpc.authorizer.acl.strict" false, "nimbus.file.copy.expiration.secs" 600, "topology.executor.receive.buffer.size" 1024, "nimbus.task.launch.secs" 120, "storm.local.mode.zmq" false, "storm.messaging.netty.buffer_size" 5242880, "worker.heartbeat.frequency.secs" 1, "ui.http.creds.plugin" "backtype.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.root" "/storm", "topology.tick.tuple.freq.secs" nil, "drpc.https.port" -1, "task.refresh.poll.secs" 10, "task.heartbeat.frequency.secs" 3, "storm.messaging.netty.max_wait_ms" 1000, "drpc.http.port" 3774, "topology.error.throttle.interval.secs" 10, "storm.messaging.transport" "backtype.storm.messaging.netty.Context", "storm.messaging.netty.authentication" false, "topology.kryo.factory" "backtype.storm.serialization.DefaultKryoFactory", "worker.gc.childopts" "", "nimbus.topology.validator" "backtype.storm.nimbus.DefaultTopologyValidator", "nimbus.cleanup.inbox.freq.secs" 600, "ui.users" nil, "transactional.zookeeper.servers" nil, "supervisor.worker.timeout.secs" 30, "storm.zookeeper.auth.password" nil, "supervisor.childopts" "-Xmx256m", "ui.filter" nil, "nimbus.authorizer" "backtype.storm.security.auth.authorizer.NoopAuthorizer", "ui.header.buffer.bytes" 4096, "storm.messaging.netty.flush.check.interval.ms" 10, "storm.nimbus.retry.intervalceiling.millis" 60000, "topology.trident.batch.emit.interval.millis" 50, "topology.disruptor.wait.strategy" "com.lmax.disruptor.BlockingWaitStrategy", "storm.auth.simple-acl.users" [], "drpc.invocations.threads" 64, "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "ui.port" 8080, "storm.messaging.netty.transfer.batch.size" 262144, "logviewer.appender.name" "A1", "nimbus.thrift.max_buffer_size" 1048576, "nimbus.host" "localhost", "storm.auth.simple-acl.users.commands" [], "drpc.request.timeout.secs" 600}
93528 [main] INFO  b.s.d.nimbus - Using default scheduler
93555 [main] INFO  b.s.u.StormBoundedExponentialBackoffRetry - The baseSleepTimeMs [1000] the maxSleepTimeMs [30000] the maxRetries [5]
93555 [main] INFO  o.a.c.f.i.CuratorFrameworkImpl - Starting
93623 [main-EventThread] INFO  o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
93841 [main-EventThread] INFO  b.s.zookeeper - Zookeeper state update: :connected:none
94898 [main] INFO  b.s.u.StormBoundedExponentialBackoffRetry - The baseSleepTimeMs [1000] the maxSleepTimeMs [30000] the maxRetries [5]
94898 [main] INFO  o.a.c.f.i.CuratorFrameworkImpl - Starting
94898 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] WARN  o.a.z.s.NIOServerCnxn - caught end of stream exception
org.apache.zookeeper.server.ServerCnxn$EndOfStreamException: Unable to read additional data from client sessionid 0x14e2d47005c0000, likely client has closed socket
	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228) [zookeeper-3.4.6.jar:3.4.6-1569965]
	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) [zookeeper-3.4.6.jar:3.4.6-1569965]
	at java.lang.Thread.run(Thread.java:745) [?:1.8.0_31]
94907 [main-EventThread] INFO  o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
94976 [main] INFO  b.s.u.StormBoundedExponentialBackoffRetry - The baseSleepTimeMs [1000] the maxSleepTimeMs [30000] the maxRetries [5]
94977 [main] INFO  o.a.c.f.i.CuratorFrameworkImpl - Starting
94992 [main-EventThread] INFO  o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
94992 [main-EventThread] INFO  b.s.zookeeper - Zookeeper state update: :connected:none
95050 [main] INFO  b.s.u.StormBoundedExponentialBackoffRetry - The baseSleepTimeMs [1000] the maxSleepTimeMs [30000] the maxRetries [5]
95059 [main] INFO  o.a.c.f.i.CuratorFrameworkImpl - Starting
95064 [main] INFO  b.s.u.StormBoundedExponentialBackoffRetry - The baseSleepTimeMs [1000] the maxSleepTimeMs [30000] the maxRetries [5]
95064 [main] INFO  o.a.c.f.i.CuratorFrameworkImpl - Starting
95115 [main-EventThread] INFO  o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
95131 [main-EventThread] INFO  o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
95138 [main-EventThread] INFO  b.s.zookeeper - Zookeeper state update: :connected:none
95149 [main] INFO  b.s.u.StormBoundedExponentialBackoffRetry - The baseSleepTimeMs [1000] the maxSleepTimeMs [30000] the maxRetries [5]
95150 [main] INFO  o.a.c.f.i.CuratorFrameworkImpl - Starting
95161 [main-EventThread] INFO  o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
95574 [main] INFO  b.s.u.Utils - Using defaults.yaml from resources
95650 [main] INFO  b.s.u.StormBoundedExponentialBackoffRetry - The baseSleepTimeMs [2000] the maxSleepTimeMs [60000] the maxRetries [0]
            
--------------------------------------------------

      Attachments

        Activity

          People

            Unassigned Unassigned
            kabhwan Jungtaek Lim
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: