[STORM-3141] NPE in WorkerState.transferLocalBatch when receiving messages for a task that isn't the first task assigned to the executor

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Fix Version/s: 2.0.0
• Component/s: storm-client
• Labels: pull-request-available
• Epic Link: Release Apache Storm 2.0.0
• Assignee: Stig Rohde Døssing
• Reporter: Stig Rohde Døssing
• Created: 02/Jul/18 18:37
• Updated: 12/Jul/18 13:55
• Resolved: 12/Jul/18 13:55

Description

2018-07-02 20:32:28.944 [Worker-Transfer] ERROR org.apache.storm.utils.Utils - Async loop died!
java.lang.NullPointerException: null
	at org.apache.storm.daemon.worker.WorkerState.transferLocalBatch(WorkerState.java:538) ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.storm.messaging.DeserializingConnectionCallback.recv(DeserializingConnectionCallback.java:71) ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.storm.messaging.local.Context$LocalClient.send(Context.java:194) ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.storm.utils.TransferDrainer.send(TransferDrainer.java:53) ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.storm.daemon.worker.WorkerTransfer.flush(WorkerTransfer.java:100) ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.storm.utils.JCQueue.consumeImpl(JCQueue.java:146) ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.storm.utils.JCQueue.consume(JCQueue.java:110) ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.storm.utils.JCQueue.consume(JCQueue.java:101) ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.storm.daemon.worker.WorkerTransfer.lambda$makeTransferThread$0(WorkerTransfer.java:82) ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.storm.utils.Utils$2.run(Utils.java:353) [storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_144]
2018-07-02 20:32:28.945 [Worker-Transfer] ERROR org.apache.storm.utils.Utils - Async loop died!
java.lang.RuntimeException: java.lang.NullPointerException
	at org.apache.storm.utils.Utils$2.run(Utils.java:368) ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_144]
Caused by: java.lang.NullPointerException
	at org.apache.storm.daemon.worker.WorkerState.transferLocalBatch(WorkerState.java:538) ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.storm.messaging.DeserializingConnectionCallback.recv(DeserializingConnectionCallback.java:71) ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.storm.messaging.local.Context$LocalClient.send(Context.java:194) ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.storm.utils.TransferDrainer.send(TransferDrainer.java:53) ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.storm.daemon.worker.WorkerTransfer.flush(WorkerTransfer.java:100) ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.storm.utils.JCQueue.consumeImpl(JCQueue.java:146) ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.storm.utils.JCQueue.consume(JCQueue.java:110) ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.storm.utils.JCQueue.consume(JCQueue.java:101) ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.storm.daemon.worker.WorkerTransfer.lambda$makeTransferThread$0(WorkerTransfer.java:82) ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	at org.apache.storm.utils.Utils$2.run(Utils.java:353) ~[storm-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
	... 1 more

When tuples are received, the method looks up which JCQueue to send to. It does this with the tuple destination task id. The map it looks in only has the queues by the starting task id of the executor's range, so if the destination is e.g. task 4 for an executor with assignment [3, 4], we hit an NPE.

Issue Links

• links to: GitHub Pull Request #2750