Uploaded image for project: 'Metron'
  1. Metron
  2. METRON-2088 Support HDP 3.1.0
  3. METRON-2241

Profiler Integration Test Fails After Storm 1.2.1 Upgrade

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Done
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None

      Description

      The ProfilerIntegrationTest fails after the upgrade to Storm 1.2.1. This is the only integration test that starts/stops multiple topologies during testing.

      -------------------------------------------------------
       T E S T S
      -------------------------------------------------------
      Running org.apache.metron.profiler.storm.integration.ProfilerIntegrationTest
      2019-08-29 16:35:49 ERROR util:0 - Async loop died!
      java.lang.RuntimeException: java.lang.IllegalStateException: This consumer has already been closed.
      	at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:522)
      	at org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:487)
      	at org.apache.storm.utils.DisruptorQueue.consumeBatch(DisruptorQueue.java:477)
      	at org.apache.storm.disruptor$consume_batch.invoke(disruptor.clj:70)
      	at org.apache.storm.daemon.executor$fn__4975$fn__4990$fn__5021.invoke(executor.clj:634)
      	at org.apache.storm.util$async_loop$fn__557.invoke(util.clj:484)
      	at clojure.lang.AFn.run(AFn.java:22)
      	at java.lang.Thread.run(Thread.java:748)
      Caused by: java.lang.IllegalStateException: This consumer has already been closed.
      	at org.apache.kafka.clients.consumer.KafkaConsumer.acquireAndEnsureOpen(KafkaConsumer.java:2202)
      	at org.apache.kafka.clients.consumer.KafkaConsumer.beginningOffsets(KafkaConsumer.java:1984)
      	at org.apache.kafka.clients.consumer.KafkaConsumer.beginningOffsets(KafkaConsumer.java:1963)
      	at org.apache.storm.kafka.spout.metrics.KafkaOffsetMetric.getValueAndReset(KafkaOffsetMetric.java:79)
      	at org.apache.storm.daemon.executor$metrics_tick$fn__4899.invoke(executor.clj:345)
      	at clojure.core$map$fn__4553.invoke(core.clj:2622)
      	at clojure.lang.LazySeq.sval(LazySeq.java:40)
      	at clojure.lang.LazySeq.seq(LazySeq.java:49)
      	at clojure.lang.RT.seq(RT.java:507)
      	at clojure.core$seq__4128.invoke(core.clj:137)
      	at clojure.core$filter$fn__4580.invoke(core.clj:2679)
      	at clojure.lang.LazySeq.sval(LazySeq.java:40)
      	at clojure.lang.LazySeq.seq(LazySeq.java:49)
      	at clojure.lang.Cons.next(Cons.java:39)
      	at clojure.lang.RT.next(RT.java:674)
      	at clojure.core$next__4112.invoke(core.clj:64)
      	at clojure.core.protocols$fn__6523.invoke(protocols.clj:170)
      	at clojure.core.protocols$fn__6478$G__6473__6487.invoke(protocols.clj:19)
      	at clojure.core.protocols$seq_reduce.invoke(protocols.clj:31)
      	at clojure.core.protocols$fn__6506.invoke(protocols.clj:101)
      	at clojure.core.protocols$fn__6452$G__6447__6465.invoke(protocols.clj:13)
      	at clojure.core$reduce.invoke(core.clj:6519)
      	at clojure.core$into.invoke(core.clj:6600)
      	at org.apache.storm.daemon.executor$metrics_tick.invoke(executor.clj:349)
      	at org.apache.storm.daemon.executor$fn__4975$tuple_action_fn__4981.invoke(executor.clj:522)
      	at org.apache.storm.daemon.executor$mk_task_receiver$fn__4964.invoke(executor.clj:471)
      	at org.apache.storm.disruptor$clojure_handler$reify__4475.onEvent(disruptor.clj:41)
      	at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:509)
      	... 7 more
      2019-08-29 16:35:49 ERROR executor:0 - 
      java.lang.RuntimeException: java.lang.IllegalStateException: This consumer has already been closed.
      	at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:522)
      	at org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:487)
      	at org.apache.storm.utils.DisruptorQueue.consumeBatch(DisruptorQueue.java:477)
      	at org.apache.storm.disruptor$consume_batch.invoke(disruptor.clj:70)
      	at org.apache.storm.daemon.executor$fn__4975$fn__4990$fn__5021.invoke(executor.clj:634)
      	at org.apache.storm.util$async_loop$fn__557.invoke(util.clj:484)
      	at clojure.lang.AFn.run(AFn.java:22)
      	at java.lang.Thread.run(Thread.java:748)
      Caused by: java.lang.IllegalStateException: This consumer has already been closed.
      	at org.apache.kafka.clients.consumer.KafkaConsumer.acquireAndEnsureOpen(KafkaConsumer.java:2202)
      	at org.apache.kafka.clients.consumer.KafkaConsumer.beginningOffsets(KafkaConsumer.java:1984)
      	at org.apache.kafka.clients.consumer.KafkaConsumer.beginningOffsets(KafkaConsumer.java:1963)
      	at org.apache.storm.kafka.spout.metrics.KafkaOffsetMetric.getValueAndReset(KafkaOffsetMetric.java:79)
      	at org.apache.storm.daemon.executor$metrics_tick$fn__4899.invoke(executor.clj:345)
      	at clojure.core$map$fn__4553.invoke(core.clj:2622)
      	at clojure.lang.LazySeq.sval(LazySeq.java:40)
      	at clojure.lang.LazySeq.seq(LazySeq.java:49)
      	at clojure.lang.RT.seq(RT.java:507)
      	at clojure.core$seq__4128.invoke(core.clj:137)
      	at clojure.core$filter$fn__4580.invoke(core.clj:2679)
      	at clojure.lang.LazySeq.sval(LazySeq.java:40)
      	at clojure.lang.LazySeq.seq(LazySeq.java:49)
      	at clojure.lang.Cons.next(Cons.java:39)
      	at clojure.lang.RT.next(RT.java:674)
      	at clojure.core$next__4112.invoke(core.clj:64)
      	at clojure.core.protocols$fn__6523.invoke(protocols.clj:170)
      	at clojure.core.protocols$fn__6478$G__6473__6487.invoke(protocols.clj:19)
      	at clojure.core.protocols$seq_reduce.invoke(protocols.clj:31)
      	at clojure.core.protocols$fn__6506.invoke(protocols.clj:101)
      	at clojure.core.protocols$fn__6452$G__6447__6465.invoke(protocols.clj:13)
      	at clojure.core$reduce.invoke(core.clj:6519)
      	at clojure.core$into.invoke(core.clj:6600)
      	at org.apache.storm.daemon.executor$metrics_tick.invoke(executor.clj:349)
      	at org.apache.storm.daemon.executor$fn__4975$tuple_action_fn__4981.invoke(executor.clj:522)
      	at org.apache.storm.daemon.executor$mk_task_receiver$fn__4964.invoke(executor.clj:471)
      	at org.apache.storm.disruptor$clojure_handler$reify__4475.onEvent(disruptor.clj:41)
      	at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:509)
      	... 7 more
      2019-08-29 16:35:49 ERROR util:0 - Halting process: ("Worker died")
      java.lang.RuntimeException: ("Worker died")
      	at org.apache.storm.util$exit_process_BANG_.doInvoke(util.clj:341)
      	at clojure.lang.RestFn.invoke(RestFn.java:423)
      	at org.apache.storm.daemon.worker$fn__5645$fn__5646.invoke(worker.clj:777)
      	at org.apache.storm.daemon.executor$mk_executor_data$fn__4860$fn__4861.invoke(executor.clj:281)
      	at org.apache.storm.util$async_loop$fn__557.invoke(util.clj:494)
      	at clojure.lang.AFn.run(AFn.java:22)
      	at java.lang.Thread.run(Thread.java:748)
      
        

      See https://api.travis-ci.org/v3/job/578435321/log.txt

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                nickwallen Nick Allen
                Reporter:
                nickwallen Nick Allen
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 10m
                  1h 10m