[SPARK-12015] Auto convert int to Double when required in pyspark.ml

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 1.5.2
• Component/s: ML, PySpark
• Labels: ml py4j pyspark type-converter
• Flags: Patch, Important
• Assignee: Unassigned
• Reporter: Jose Antonio
• Created: 26/Nov/15 14:00
• Updated: 17/May/17 13:11
• Resolved: 17/May/17 13:11

Description

I have received the following exception:

Why we cannot pass an integer when the required parameter is Double?

I think this should be upcasted silently.

java.lang.IllegalArgumentException: requirement failed: Column label must be of type DoubleType but was actually IntegerType.
at scala.Predef$.require(Predef.scala:233)
at org.apache.spark.ml.util.SchemaUtils$.checkColumnType(SchemaUtils.scala:42)
at org.apache.spark.ml.PredictorParams$class.validateAndTransformSchema(Predictor.scala:53)
at org.apache.spark.ml.classification.Classifier.org$apache$spark$ml$classification$ClassifierParams$$super$validateAndTransformSchema(Classifier.scala:56)

Issue Links

• duplicates: SPARK-7425 spark.ml Predictor should support other numeric types for label