Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-11939

PySpark support model export/import for Pipeline API

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Umbrella
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • None
    • 2.0.0
    • ML, PySpark
    • None

    Description

      SPARK-6725 provide model export/import for Pipeline API at Scala side, we should also support it at Python side.

      Attachments

        Issue Links

        contains

        New Feature - A new feature of the product, which has yet to be developed. SPARK-21221 CrossValidator and TrainValidationSplit Persist Nested Estimators such as OneVsRest

        • Major - Major loss of function.
        • Resolved
        is related to

        New Feature - A new feature of the product, which has yet to be developed. SPARK-17025 Cannot persist PySpark ML Pipeline model that includes custom Transformer

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Resolved
        is required by

        Test - A new unit, integration or system test. SPARK-14706 Python ML persistence integration test

        • Major - Major loss of function.
        • Resolved
        relates to

        Umbrella - An overarching type made of sub-tasks SPARK-6725 Model export/import for Pipeline API (Scala)

        • Critical - Crashes, loss of data, severe memory leak.
        • Resolved
        1.
        		 Basic ML Pipeline export/import functions for PySpark Sub-task Resolved Yanbo Liang Actions
        2.
        		 PySpark ml.regression support export/import Sub-task Resolved Wenpei Yu Actions
        3.
        		 PySpark ml.classification support export/import Sub-task Resolved Gayathri Murali Actions
        4.
        		 PySpark ml.clustering support export/import Sub-task Resolved Yanbo Liang Actions
        5.
        		 PySpark ml.feature support export/import Sub-task Resolved Xusen Yin Actions
        6.
        		 PySpark ml.recommendation support export/import Sub-task Resolved Kai Jiang Actions
        7.
        		 PySpark ml.pipeline support export/import - non-nested Pipelines Sub-task Resolved Xusen Yin Actions
        8.
        		 Pyspark ml.tuning support export/import Sub-task Resolved Unassigned Actions
        9.
        		 PySpark ml DecisionTreeClassifier, Regressor support export/import Sub-task Resolved Gayathri Murali Actions
        10.
        		 PySpark ml.pipeline support export/import - nested Piplines Sub-task Resolved Xusen Yin Actions
        11.
        		 PySpark ml.feature.RFormula/RFormulaModel support export/import Sub-task Resolved Xusen Yin Actions
        12.
        		 Change MLWritable.write to be a property Sub-task Resolved Miao Wang Actions
        13.
        		 MultilayerPerceptronClassifier supports save/load for Python API Sub-task Resolved Yanbo Liang Actions
        14.
        		 PySpark ml.clustering BisectingKMeans support export/import Sub-task Resolved Yanbo Liang Actions
        15.
        		 PySpark ml.classification OneVsRest support export/import Sub-task Resolved Xusen Yin Actions
        16.
        		 PySpark ml RandomForestClassifier, Regressor support export/import Sub-task Resolved Kai Jiang Actions
        17.
        		 PySpark ml GBTClassifier, Regressor support export/import Sub-task Resolved Yanbo Liang Actions

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            josephkb Joseph K. Bradley
            yanboliang Yanbo Liang
            Votes:
            1 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment