Description
Currently, most Python models only have limited support for single-vector prediction.
E.g., one can call
model.predict(myFeatureVector)
for a single instance, but that fails within a map for Python ML models and transformers which use JavaModelWrapper:
data.map(lambda features: model.predict(features))
This fails because JavaModelWrapper.call uses the SparkContext (within the transformation). (It works for linear models, which do prediction within Python.)
Supporting prediction within a map would require storing the model and doing prediction/transformation within Python.
Attachments
Issue Links
- contains
-
SPARK-9122 spark.mllib regression should support batch predict
-
- Resolved
-