Details
Major Description
When I fit a LogisticRegression on a dataset, everything works fine but when I fit a CrossValidator, I get this error:
py4j.protocol.Py4JJavaError: An error occurred while calling o1187.w.
: java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.Double
at scala.runtime.BoxesRunTime.unboxToDouble(BoxesRunTime.java:114)
at org.apache.spark.ml.param.DoubleParam.w(params.scala:330)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
at py4j.Gateway.invoke(Gateway.java:282)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:238)
at java.lang.Thread.run(Unknown Source)
Casting the target variable into double didn't solve the issue.
Here is the snippet:
lr = LogisticRegression(maxIter=10, labelCol="class", featuresCol="features", rawPredictionCol="score")
evaluator = BinaryClassificationEvaluator(rawPredictionCol="score", labelCol="class", metricName="areaUnderROC")
paramGrid = ParamGridBuilder().addGrid(lr.regParam, [0.01, 0.05, 0.1, 0.5, 1]).build()
crossval = CrossValidator(estimator=lr, estimatorParamMaps=paramGrid, evaluator=evaluator, numFolds=3)
bestModel = crossval.fit(train)
