[SPARK-19400] GLM fails for intercept only model

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.2.0
• Component/s: ML
• Assignee: Wayne Zhang
• Reporter: Wayne Zhang
• Shepherd: Yanbo Liang
• Created: 30/Jan/17 08:07
• Updated: 08/Feb/17 18:42
• Resolved: 08/Feb/17 18:42

Description

Intercept-only GLM fails for non-Gaussian family because of reducing an empty array in IWLS.

val dataset = Seq(
          (1.0, 1.0, 2.0, 0.0, 5.0),
          (0.5, 2.0, 1.0, 1.0, 2.0),
          (1.0, 3.0, 0.5, 2.0, 1.0),
          (2.0, 4.0, 1.5, 3.0, 3.0)
        ).toDF("y", "w", "off", "x1", "x2")

val formula = new RFormula().setFormula("y ~ 1")
val output = formula.fit(dataset).transform(dataset)
val glr = new GeneralizedLinearRegression().setFamily("poisson")
val model = glr.fit(output)

java.lang.UnsupportedOperationException: empty.reduceLeft

Issue Links

• links to: [Github] Pull Request #16740 (actuaryzhang)