[SPARK-10119] Utils.isDynamicAllocationEnabled returns wrong value if dynAlloc is explicitly disabled

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.5.0
• Fix Version/s: 1.5.0
• Component/s: Spark Core
• Target Version/s: 1.5.0
• Assignee: Marcelo Masiero Vanzin
• Reporter: Marcelo Masiero Vanzin
• Created: 19/Aug/15 18:13
• Updated: 19/Aug/15 21:34
• Resolved: 19/Aug/15 21:34

Description

  def isDynamicAllocationEnabled(conf: SparkConf): Boolean = {
    conf.contains("spark.dynamicAllocation.enabled") &&
      conf.getInt("spark.executor.instances", 0) == 0
  }

If you have spark.dynamicAllocation.enabled=false in the config, that will return true.

Issue Links

• links to: [Github] Pull Request #8316 (vanzin)