[MESOS-7829] Improve completed task/framework garbage collection

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: master
• Labels: master mesosphere
• Story Points: 5
• Assignee: Unassigned
• Reporter: Greg Mann
• Created: 25/Jul/17 18:48
• Updated: 04/Aug/17 15:21

Description

The Mesos master currently uses two flags to determine how it garbage collects completed tasks and frameworks from memory:

• --max_completed_frameworks
• --max_completed_tasks_per_framework

Setting these parameters correctly can be difficult, since there may be a large variance in the size of Task and Framework objects kept in memory. Launching a framework which makes use of task labels to pass data of significant size can quickly lead to performance issues if the master is retaining a large number of completed tasks.

We should explore other ways of garbage collecting completed frameworks and tasks, which could better handle the variation in the size of task metadata.

Issue Links

• is related to: MESOS-7018 Move historical data out of memory.