Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-17133 Improvements to linear methods in Spark
  3. SPARK-17090

Make tree aggregation level in linear/logistic regression configurable

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • 2.1.0
    • ML
    • None

    Description

      Linear/logistic regression use treeAggregate with default aggregation depth for collecting coefficient gradient updates to the driver. For high dimensional problems, this can case OOM error on the driver. We should make it configurable, perhaps via an expert param, so that users can avoid this problem if their data has many features.

      Attachments

        Issue Links

          relates to

          Task - A task that needs to be done. SPARK-17175 Add a expert formula to aggregationDepth of SharedParam

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          links to

          Pull request #14717 [Github] Pull Request #14717 (hqzizania)

          Pull request #14738 [Github] Pull Request #14738 (hqzizania)

          Activity

            People

              qhuang Qian Huang
              sethah Seth Hendrickson
              Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: