[MESOS-10056] Perform synchronous authorization for scheduler calls.

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.10.0
• Component/s: master
• Assignee: Andrei Sekretenko
• Reporter: Benjamin Mahler
• Created: 30/Nov/19 17:18
• Updated: 03/Mar/20 14:45
• Resolved: 03/Mar/20 14:24

Description

After chatting with asekretenko about how best to resolve MESOS-10023, as an alternative to making all scheduler calls get sequenced through an asynchronous authorization step, I brought up the old idea of making authorization synchronous.

This came up (although I can't find a ticket for it) in the past because the master event stream outgoing message authorization becomes very expensive for a large number of subscribers (cc greggomann). Back then, I suggested that we always hold on to valid object approvers so that we could synchronously (and cheaply) authorize the outgoing events. These object approvers would be kept up to date in the background, and if authorization fails to keep them up to date, we would treat that the same as an authorization failure is currently treated.

We can apply the same idea (although we haven't applied it to the master's event stream yet) to scheduler API calls, which should help resolve MESOS-10023 since we're no longer mixing asynchronously authorized calls with calls that don't go through authorization.

This will also yield a performance improvement, scheduler calls no longer get delayed by asynchronous authorization, and an extra trip through the master queue.

Issue Links

• blocks: MESOS-10057 Perform synchronous authorization for outgoing events on event stream.
• fixes: MESOS-7639 Oversubscription could crash the master due to CHECK failure in the allocator
• fixes: MESOS-10023 Allocator method dispatches can be reordered (relative to scheduler API calls which triggered them).
• fixes: MESOS-10083 Authorizing invalid operation can result in declined authorization.
• relates to: MESOS-10099 Operator API can silently drop objects due to authorization errors.
• relates to: MESOS-10083 Authorizing invalid operation can result in declined authorization.
• relates to: MESOS-10097 After HTTP framework disconnects, heartbeater idle-loops instead of being deleted.
• links to: Design proposal: synchronous authorization in Mesos