[MESOS-7564] Introduce a heartbeat mechanism for v1 HTTP executor <-> agent communication.

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Done
• Fix Version/s: 1.8.0
• Component/s: agent, executor
• Labels: api foundations mesosphere v1_api
• Epic Link: Make the HTTP command executor the default implementation.
• Sprint: Mesos Foundation R7 Sprint 33, Mesos Foundations R8 Sprint 34, Mesos Foundations R8 Sprint 35
• Story Points: 5
• Assignee: Joseph Wu
• Reporter: Anand Mazumdar
• Created: 25/May/17 00:24
• Updated: 11/Apr/19 00:09
• Resolved: 14/Dec/18 21:38

Description

Currently, we do not have heartbeats for executor <-> agent communication. This is especially problematic in scenarios when IPFilters are enabled since the default conntrack keep alive timeout is 5 days. When that timeout elapses, the executor doesn't get notified via a socket disconnection when the agent process restarts. The executor would then get killed if it doesn't re-register when the agent recovery process is completed.

Enabling application level heartbeats or TCP KeepAlive's can be a possible way for fixing this issue.

We should also update executor API documentation to explain the new behavior.

Issue Links

• is cloned by: MESOS-7568 Introduce a heartbeat mechanism for v0 executor <-> agent links.
• is related to: MESOS-9727 Heartbeat calls from executor to agent are reported as errors
• is related to: MESOS-5361 Consider introducing TCP KeepAlive for Libprocess sockets.
• relates to: MESOS-8366 Replace the command executor with the default executor.
• relates to: MESOS-540 Executor health checking.
• relates to: MESOS-9258 Prevent subscribers to the master's event stream from leaking connections