Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-22422

Add Adjusted R2 to RegressionMetrics

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.2.0
    • Fix Version/s: 2.3.0
    • Component/s: ML
    • Labels:
      None

      Description

      In practice, no one looks at R2 alone. The reason is R2 itself is misleading. If we add more parameters, R2 will not decrease but only increase (or stay the same). This leads to overfitting.

      I added adjusted R2 as the metric which was implemented in all major statistical analysis tools.

        Attachments

          Activity

            People

            • Assignee:
              Teng Peng Teng Peng
              Reporter:
              Teng Peng Teng Peng
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: