[SPARK-13371] TaskSetManager.dequeueSpeculativeTask compares Option[String] and String directly.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.5.2, 1.6.0
• Fix Version/s: 1.6.1, 2.0.0
• Component/s: Scheduler
• Target Version/s: 1.6.1, 2.0.0
• Assignee: Sean R. Owen
• Reporter: Guoqiang Li
• Created: 18/Feb/16 05:47
• Updated: 18/Feb/16 20:14
• Resolved: 18/Feb/16 20:14

Description

TaskSetManager.dequeueSpeculativeTask compares Option[String] and String directly.

Ths code: https://github.com/apache/spark/blob/87abcf7df921a5937fdb2bae8bfb30bfabc4970a/core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala#L344

 if (TaskLocality.isAllowed(locality, TaskLocality.RACK_LOCAL)) {
    for (rack <- sched.getRackForHost(host)) {
      for (index <- speculatableTasks if canRunOnHost(index)) {
        val racks = tasks(index).preferredLocations.map(_.host).map(sched.getRackForHost)
        // racks: Seq[Option[String]] and rack: String
        if (racks.contains(rack)) {
          speculatableTasks -= index
          return Some((index, TaskLocality.RACK_LOCAL))
        }
      }
    }
  }

Issue Links

• links to: [Github] Pull Request #11253 (srowen)