[SPARK-38083] set the amount of explained variance as parameter of pyspark.ml.feature.PCA

• Type: Wish
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.2.2
• Component/s: ML, MLlib
• Assignee: Unassigned
• Reporter: Nicola
• Created: 01/Feb/22 17:38
• Updated: 01/Feb/22 17:43

Description

As in sklearn.decomposition.PCA, where:

if 0 < n_components < 1 select the number of components such that the amount of variance that needs to be explained is greater than the percentage specified by n_components

it would be useful to have a similar behavior with the k parameter in pyspark.ml.feature.PCA.