[SPARK-10444] Remove duplication in Mesos schedulers

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 1.5.0
• Component/s: Mesos
• Labels: bulk-closed refactoring
• Assignee: Unassigned
• Reporter: Dragos Dascalita Haut
• Created: 04/Sep/15 08:31
• Updated: 21/May/19 04:35
• Resolved: 21/May/19 04:35

Description

Currently coarse-grained and fine-grained Mesos schedulers don't share much code, and that leads to inconsistencies. For instance:

• only coarse-grained mode respects spark.cores.max, see SPARK-9873
• only coarse-grained mode blacklists slaves that fail repeatedly, but that seams like generally useful
• constraints and memory checking are done on both sides (code is shared though)
• framework re-registration (master election) is only done for cluster-mode deployment

We should find a better design that groups together common concerns and generally improves the code.