[STORM-770] NullPointerException in consumeBatchToCursor

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.9.2-incubating
• Component/s: storm-core
• Assignee: Unassigned
• Reporter: Stas Levin
• Created: 14/Apr/15 08:55
• Updated: 05/Oct/15 01:15

Description

We got the following exception after our topology had been up for ~2 days, and I was wondering if it might be related.
Looks like "task" in "mk-transfer-fn" is null, making "(.add remote (TaskMessage. task (.serialize serializer tuple)))" fail on NPE (worker.clj:128, storm-core-0.9.2-incubating.jar)

java.lang.RuntimeException: java.lang.NullPointerException
at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:128) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at backtype.storm.disruptor$consume_loop_STAR_$fn__758.invoke(disruptor.clj:94) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at backtype.storm.util$async_loop$fn__457.invoke(util.clj:431) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_72]
Caused by: java.lang.NullPointerException: null
at clojure.lang.RT.intCast(RT.java:1087) ~[clojure-1.5.1.jar:na]
at backtype.storm.daemon.worker$mk_transfer_fn$fn__5748.invoke(worker.clj:128) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at backtype.storm.daemon.executor$start_batch_transfer_GT_worker_handler_BANG$fn__5483.invoke(executor.clj:256) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at backtype.storm.disruptor$clojure_handler$reify__745.onEvent(disruptor.clj:58) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
... 6 common frames omitted,java.lang.RuntimeException: java.lang.NullPointerException

Any ideas?

P.S.
Also saw it here:
http://mail-archives.apache.org/mod_mbox/storm-user/201501.mbox/%3CCABcMBhCusXXU=[e-mail]%3E

https://mail-archives.apache.org/mod_mbox/storm-user/201408.mbox/[e-mail]%3E

Comment from Bobby
http://mail-archives.apache.org/mod_mbox/storm-user/201501.mbox/[e-mail]%3E

What version of storm are you using? Are any of the bolts shell bolts? There is a known
issue where this can happen if two shell bolts share an executor, because they are multi-threaded.

• Bobby

Issue Links

• is related to: STORM-841 Thread-safeness of OutputCollector has documented contrary to two official doc.
• relates to: STORM-737 Workers may try to send to closed connections
• relates to: STORM-790 Log "task id is null" instead of let worker died (NPE in consumeBatchToCursor)