[AURORA-1228] Use explicit status update acknowledgements to improve driver throughput.

• Type: Task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.9.0
• Component/s: Scheduler
• Sprint: Twitter Aurora Q2'15 Sprint 3
• Assignee: Benjamin Mahler
• Reporter: Benjamin Mahler
• Created: 25/Mar/15 17:50
• Updated: 20/Jul/15 16:43
• Resolved: 14/May/15 01:12

Description

Mesos 0.22.0 is released and includes the ability for schedulers to perform explicit acknowledgments of status updates.

This allows schedulers to process status updates asynchronously from the driver thread. Updates can then be processed in batches to reduce the per-update processing overhead. These should help provide substantial improvements to the schedulers update processing throughput.

I've placed this under the GC executor retirement effort, per AURORA-1047.

Issue Links

• blocks: AURORA-1047 Implement state reconciliation within the scheduler
• is blocked by: AURORA-1276 Upgrade to mesos 0.22
• is blocked by: AURORA-1283 Implement a benchmark for status update processing throughput.