Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-19400

GLM fails for intercept only model

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.2.0
    • Component/s: ML
    • Labels:
      None

      Description

      Intercept-only GLM fails for non-Gaussian family because of reducing an empty array in IWLS.

      val dataset = Seq(
                (1.0, 1.0, 2.0, 0.0, 5.0),
                (0.5, 2.0, 1.0, 1.0, 2.0),
                (1.0, 3.0, 0.5, 2.0, 1.0),
                (2.0, 4.0, 1.5, 3.0, 3.0)
              ).toDF("y", "w", "off", "x1", "x2")
      
      val formula = new RFormula().setFormula("y ~ 1")
      val output = formula.fit(dataset).transform(dataset)
      val glr = new GeneralizedLinearRegression().setFamily("poisson")
      val model = glr.fit(output)
      
      java.lang.UnsupportedOperationException: empty.reduceLeft
      

        Attachments

          Activity

            People

            • Assignee:
              actuaryzhang Wayne Zhang
              Reporter:
              actuaryzhang Wayne Zhang
              Shepherd:
              Yanbo Liang
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: