Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-3668

Apache Spark Java Quickstart fails 2.3.0 RC2

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: P0
    • Resolution: Fixed
    • Affects Version/s: 2.3.0
    • Fix Version/s: 2.3.0
    • Component/s: examples-java, runner-spark
    • Labels:
      None

      Description

      Running the quickstart command fails:

      mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
           -Dexec.args="--runner=SparkRunner --inputFile=pom.xml --output=counts" -Pspark-runner
      

      with:

      SEVERE: Error sending result StreamResponse{streamId=/jars/json4s-ast_2.11-3.2.11.jar, byteCount=82421, body=FileSegmentManagedBuffer{file=/usr/local/google/home/lcwik/.m2/repository/org/json4s/json4s-ast_2.11/3.2.11/json4s-ast_2.11-3.2.11.jar, offset=0, length=82421}} to /100.100.214.107:48220; closing connection
      java.lang.AbstractMethodError
      at io.netty.util.ReferenceCountUtil.touch(ReferenceCountUtil.java:73)
      at io.netty.channel.DefaultChannelPipeline.touch(DefaultChannelPipeline.java:107)
      at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:811)
      at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:724)
      at io.netty.handler.codec.MessageToMessageEncoder.write(MessageToMessageEncoder.java:111)
      at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:739)
      at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:731)
      at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:817)
      at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:724)
      at io.netty.handler.timeout.IdleStateHandler.write(IdleStateHandler.java:305)
      at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:739)
      at io.netty.channel.AbstractChannelHandlerContext.invokeWriteAndFlush(AbstractChannelHandlerContext.java:802)
      at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:815)
      at io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:795)
      at io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:832)
      at io.netty.channel.DefaultChannelPipeline.writeAndFlush(DefaultChannelPipeline.java:1032)
      at io.netty.channel.AbstractChannel.writeAndFlush(AbstractChannel.java:296)
      at org.apache.spark.network.server.TransportRequestHandler.respond(TransportRequestHandler.java:192)
      at org.apache.spark.network.server.TransportRequestHandler.processStreamRequest(TransportRequestHandler.java:148)
      at org.apache.spark.network.server.TransportRequestHandler.handle(TransportRequestHandler.java:109)
      at org.apache.spark.network.server.TransportChannelHandler.channelRead(TransportChannelHandler.java:118)
      at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
      at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:349)
      at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:341)
      at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287)
      at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
      at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:349)
      at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:341)
      at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
      at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
      at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:349)
      at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:341)
      at org.apache.spark.network.util.TransportFrameDecoder.channelRead(TransportFrameDecoder.java:85)
      at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
      at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:349)
      at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:341)
      at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334)
      at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
      at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:349)
      at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926)
      at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:129)
      at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:642)
      at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:565)
      at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:479)
      at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:441)
      at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
      at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
      at java.lang.Thread.run(Thread.java:745)

        Attachments

        1. full_log.txt
          134 kB
          Luke Cwik

          Issue Links

            Activity

              People

              • Assignee:
                jbonofre Jean-Baptiste Onofré
                Reporter:
                lcwik Luke Cwik
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 2h 20m
                  2h 20m