[SPARK-19133] SparkR glm Gamma family results in error

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.0, 2.1.0
• Fix Version/s: 2.0.3, 2.1.1, 2.2.0
• Component/s: ML, SparkR
• Target Version/s: 2.0.3, 2.1.1, 2.2.0
• Assignee: Felix Cheung
• Reporter: Felix Cheung
• Created: 09/Jan/17 06:07
• Updated: 12/Jan/17 04:04
• Resolved: 10/Jan/17 19:49

Description

> glm(y~1,family=Gamma, data = dy)
17/01/09 06:10:47 ERROR RBackendHandler: fit on org.apache.spark.ml.r.GeneralizedLinearRegressionWrapper failed
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.api.r.RBackendHandler.handleMethodCall(RBackendHandler.scala:167)
at org.apache.spark.api.r.RBackendHandler.channelRead0(RBackendHandler.scala:108)
at org.apache.spark.api.r.RBackendHandler.channelRead0(RBackendHandler.scala:40)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:367)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:353)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:346)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:266)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:367)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:353)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:346)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:367)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:353)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:346)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:293)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:267)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:367)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:353)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:346)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1294)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:367)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:353)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:911)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:652)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:575)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:489)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:451)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:140)
at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: glm_e3483764cdf9 parameter family given invalid value Gamma.
at org.apache.spark.ml.param.Param.validate(params.scala:77)
at org.apache.spark.ml.param.ParamPair.<init>(params.scala:528)
at org.apache.spark.ml.param.Param.$minus$greater(params.scala:87)
at org.apache.spark.ml.param.Params$class.set(params.scala:609)
at org.apache.spark.ml.PipelineStage.set(Pipeline.scala:42)
at org.apache.spark.ml.regression.GeneralizedLinearRegression.setFamily(GeneralizedLinearRegression.scala:157)
at org.apache.spark.ml.r.GeneralizedLinearRegressionWrapper$.fit(GeneralizedLinearRegressionWrapper.scala:85)
at org.apache.spark.ml.r.GeneralizedLinearRegressionWrapper.fit(GeneralizedLinearRegressionWrapper.scala)
... 36 more
Error in handleErrors(returnStatus, conn) :
java.lang.IllegalArgumentException: glm_e3483764cdf9 parameter family given invalid value Gamma.
at org.apache.spark.ml.param.Param.validate(params.scala:77)
at org.apache.spark.ml.param.ParamPair.<init>(params.scala:528)
at org.apache.spark.ml.param.Param.$minus$greater(params.scala:87)
at org.apache.spark.ml.param.Params$class.set(params.scala:609)
at org.apache.spark.ml.PipelineStage.set(Pipeline.scala:42)
at org.apache.spark.ml.regression.GeneralizedLinearRegression.setFamily(GeneralizedLinearRegression.scala:157)
at org.apache.spark.ml.r.GeneralizedLinearRegressionWrapper$.fit(GeneralizedLinearRegressionWrapper.scala:85)
at org.apache.spark.ml.r.GeneralizedLinearRegressionWrapper.fit(GeneralizedLinearRegressionWrapper.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

Issue Links

• links to: [Github] Pull Request #16511 (felixcheung)
• links to: [Github] Pull Request #16532 (felixcheung)
• links to: [Github] Pull Request #16543 (felixcheung)