[MESOS-6660] Status updates and disconnected frameworks

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: master
• Labels: mesosphere
• Assignee: Unassigned
• Reporter: Neil Conway
• Created: 01/Dec/16 15:07
• Updated: 01/Dec/16 15:09

Description

When the master receives a status update, it invokes Master::forward to send the status update to the framework. Master::forward also updates the task's status_update_state and status_update_uuid fields. However, Master::forward is not invoked for disconnected frameworks.

This scheme has the following drawbacks:

1. Logically, Master::forward probably shouldn't be updating the state of a task; forward should just forward messages.
2. The reasoning for not updating status_update_state and status_update_uuid for disconnected frameworks is to try to avoid inconsistencies upon framework reconciliation – we don't want reconciliation to show update X, but then the framework to receive a non-reconciliation update Y, such that Y comes before X in the agent's order. I'm not sure that this is actually a problem (since we wait for framework acks before forwarding subsequent status updates), but in any case, depending on framework connectivity cannot be a correct fix: whether the master thinks the framework is connected at a given time is unreliable and racy (i.e., the master might think the framework is disconnected but it still might receive messages; similarly, "connected" frameworks might not receive messages).