[STORM-2213] ShellSpout has race condition when ShellSpout is being inactive longer than heartbeat timeout

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.0.0, 1.1.0, 1.0.3
• Component/s: storm-multilang
• Assignee: Jungtaek Lim
• Reporter: Jungtaek Lim
• Created: 22/Nov/16 08:52
• Updated: 24/Nov/16 15:21
• Resolved: 24/Nov/16 15:21

Description

I brought additional guard logic for long idle ShellSpout (STORM-1928), which resolves the issue, but I found race condition from new logic.

Scenario:

1. lastHeartbeat is not updated more than timeout because of inactivity
2. querySubprocess() is called
3. waitingOnSubprocess is set to true
4. HeartbeatTimerTask.run() triggers faster than updating heartbeat (getting message from subprocess)

Simplest approach is updating heartbeat before set waitingOnSubprocess to true. Last heartbeat time is no longer only from subprocess, but it doesn't harm.

Issue Links

• links to: GitHub Pull Request #1792