[SPARK-2557] createTaskScheduler should be consistent between local and local-n-failures

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.0.0
• Fix Version/s: 1.1.0
• Component/s: Spark Core
• Labels: starter
• Assignee: Unassigned
• Reporter: YE
• Created: 17/Jul/14 15:10
• Updated: 17/Sep/15 20:40
• Resolved: 01/Aug/14 07:35

Description

In SparkContext.createTaskScheduler, we can use

local[*]

to estimates the number of cores on the machine. I think we should also be able to use * in the local-n-failures mode.

And according to the code in the LOCAL_N_REGEX pattern matching code, I believe the regular expression of LOCAL_N_REGEX is wrong. LOCAL_N_REFEX should be

"""local\[([0-9]+|\*)\]""".r

rather than

 """local\[([0-9\*]+)\]""".r