[SPARK-32455] LogisticRegressionModel prediction optimization

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.1.0
• Fix Version/s: 3.1.0
• Component/s: ML
• Assignee: Ruifeng Zheng
• Reporter: Ruifeng Zheng
• Created: 27/Jul/20 08:58
• Updated: 12/Oct/20 08:24
• Resolved: 30/Jul/20 02:54

Description

if needed, method getThreshold and/or following logic to compute rawThreshold is called on each instance.

 

override def getThreshold: Double = {
  checkThresholdConsistency()
  if (isSet(thresholds)) {
    val ts = $(thresholds)
    require(ts.length == 2, "Logistic Regression getThreshold only applies to" +
      " binary classification, but thresholds has length != 2.  thresholds: " + ts.mkString(","))
    1.0 / (1.0 + ts(0) / ts(1))
  } else {
    $(threshold)
  }
} 

 

      val rawThreshold = if (t == 0.0) {
            Double.NegativeInfinity
          } else if (t == 1.0) {
            Double.PositiveInfinity
          } else {
            math.log(t / (1.0 - t))
          } 

Issue Links

• links to: [Github] Pull Request #29255 (zhengruifeng)
• links to: [Github] Pull Request #29255 (zhengruifeng)
• links to: [Github] Pull Request #30013 (zhengruifeng)
• links to: [Github] Pull Request #30013 (zhengruifeng)