Description
hi,
i am getting java.lang.AssertionError error when running glm, using gaussian link function, on a dataset with 109 columns and 81318461 rows
Below is the call trace. Can someone please tell me what the issues is related to and how to go about resolving it. Is it because native acceleration is not working as i am also seeing following warning messages.
WARN netlib.BLAS: Failed to load implementation from: com.github.fommil.netlib.NativeRefBLAS
WARN netlib.LAPACK: Failed to load implementation from: com.github.fommil.netlib.NativeSystemLAPACK
WARN netlib.LAPACK: Failed to load implementation from: com.github.fommil.netlib.NativeRefLAPACK
16/09/17 13:08:13 ERROR r.RBackendHandler: fit on org.apache.spark.ml.r.GeneralizedLinearRegressionWrapper failed
Error in invokeJava(isStatic = TRUE, className, methodName, ...) :
java.lang.AssertionError: assertion failed: lapack.dppsv returned 105.
at scala.Predef$.assert(Predef.scala:170)
at org.apache.spark.mllib.linalg.CholeskyDecomposition$.solve(CholeskyDecomposition.scala:40)
at org.apache.spark.ml.optim.WeightedLeastSquares.fit(WeightedLeastSquares.scala:140)
at org.apache.spark.ml.regression.GeneralizedLinearRegression.train(GeneralizedLinearRegression.scala:265)
at org.apache.spark.ml.regression.GeneralizedLinearRegression.train(GeneralizedLinearRegression.scala:139)
at org.apache.spark.ml.Predictor.fit(Predictor.scala:90)
at org.apache.spark.ml.Predictor.fit(Predictor.scala:71)
at org.apache.spark.ml.Pipeline$$anonfun$fit$2.apply(Pipeline.scala:149)
at org.apache.spark.ml.Pipeline$$anonfun$fit$2.apply(Pipeline.scala:145)
at scala.collection.Iterator$class.foreach(Iterator.scala:893)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
at scala.collection.IterableViewLike$Transformed$class.foreach(IterableViewLike.sc
thanks,
pavan.
Attachments
Issue Links
- duplicates
-
SPARK-11918 Better error from WLS for cases like singular input
-
- Resolved
-