[STORM-871] Change multilang heartbeat mechanism to multi-threading of subprocess

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Component/s: storm-multilang
• Assignee: Jungtaek Lim
• Reporter: Jungtaek Lim
• Created: 17/Jun/15 21:17
• Updated: 29/Sep/15 02:46
• Resolved: 15/Jul/15 13:19

Description

Having design constraint of multilang heartbeat, some issues are raised at the moment.
(https://issues.apache.org/jira/browse/STORM-738 is a kind of reporting)

I'm trying to add workarounds (https://issues.apache.org/jira/browse/STORM-742 and so on), but if we can get rid of design constraint it would be really great.

Maybe we can use pid file of subprocess (subprocess write it from sendpid()) to check subprocess heartbeat from ShellSpout / ShellBolt.
Letting subprocess launch new thread which writes pid file periodically, to make sure "modified time" of file is periodically up to date. (It is just an idea, and it should be validated.)

Issue Links

• contains: STORM-742 Very busy ShellBolt subprocess with ACK mode cannot respond heartbeat just in time
• contains: STORM-758 Very busy ShellBolt subprocess with Non-ACK mode cannot respond heartbeat just in time