[SPARK-11937] Python API coverage check found issues for ML during 1.6 QA

• Type: Umbrella
• Status: Closed
• Priority: Major
• Resolution: Done
• Component/s: Documentation, ML, MLlib, PySpark
• Assignee: Unassigned
• Reporter: Yanbo Liang
• Created: 24/Nov/15 03:06
• Updated: 25/May/16 20:34
• Resolved: 25/May/16 20:34

Description

Here is the todo list of SPARK-11604 found issues:
Note: I did not list the SparkR related features (such as ml.feature.Interaction). We have supported RFormula as a wrapper at Python side, I think we should discuss the necessary to support other R related features at Python side.

• Missing classes
  • ml.attribute SPARK-8516
  • ml.feature
    • QuantileDiscretizer SPARK-11922
    • ChiSqSelector SPARK-11923
  • ml.classification
    • OneVsRest SPARK-7861
  • ml.clustering
    • LDA SPARK-11940
  • mllib.clustering
    • BisectingKMeans SPARK-11944

• Missing methods/parameters SPARK-11938
  • ml.classification SPARK-11815 SPARK-11820
  • ml.feature SPARK-11925
  • ml.clustering SPARK-11945
  • mllib.linalg SPARK-12040 SPARK-12041
  • mllib.stat.test.StreamingTest SPARK-12042

• Docs:
  • ml.classification SPARK-11875

Issue Links

• is related to: SPARK-11604 ML 1.6 QA: API: Python API coverage
• relates to: SPARK-12810 PySpark CrossValidatorModel should support avgMetrics

Sub-Tasks

• Python API for ml.feature.QuantileDiscretizer: Resolved
• Update doc for PySpark HasCheckpointInterval: Resolved
• PySpark LiR & LoR should support weightCol: Resolved
• Add PySpark missing methods for ml.feature during Spark 1.6 QA: Resolved
• Python API for ml.feature.ChiSqSelector: Resolved
• PySpark DecisionTreeClassifier & DecisionTreeRegressor should support setSeed: Resolved
• Expose numFeatures in all ML PredictionModel for PySpark: Resolved
• Python API for ml.clustering.LDA: Resolved
• Python API for mllib.clustering.BisectingKMeans: Resolved
• Add computeCost to KMeansModel for PySpark spark.ml: Resolved
• Add toJson/fromJson to Vector/Vectors for PySpark: Closed
• Add columnSimilarities to IndexedRowMatrix for PySpark: Resolved
• Feature parity for pyspark.mllib StandardScalerModel: Resolved