[SPARK-21115] If the cores left is less than the coresPerExecutor,the cores left will not be allocated, so it should not to check in every schedule

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.1.1
• Fix Version/s: 2.3.0
• Component/s: Spark Core
• Assignee: eaton
• Reporter: eaton
• Created: 16/Jun/17 06:00
• Updated: 23/Jun/17 12:54
• Resolved: 23/Jun/17 12:54

Description

If we start an app with the param --total-executor-cores=4 and spark.executor.cores=3, the cores left is always 1, so it will try to allocate executors in the function org.apache.spark.deploy.master.startExecutorsOnWorkers in every schedule.
Another question is, is it will be better to allocate another executor with 1 core for the cores left.

Issue Links

• links to: [Github] Pull Request #18322 (eatoncys)