Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-7864

Portable Spark Reshuffle coder cast exception

    XMLWordPrintableJSON

    Details

      Description

      running :sdks:python:test-suites:portable:py35:portableWordCountBatch in either loopback or docker mode on master fails with exception:

       

      java.lang.ClassCastException: org.apache.beam.sdk.coders.LengthPrefixCoder cannot be cast to org.apache.beam.sdk.coders.KvCoder
      at org.apache.beam.runners.spark.translation.SparkBatchPortablePipelineTranslator.translateReshuffle(SparkBatchPortablePipelineTranslator.java:400)
      at org.apache.beam.runners.spark.translation.SparkBatchPortablePipelineTranslator.translate(SparkBatchPortablePipelineTranslator.java:147)
      at org.apache.beam.runners.spark.SparkPipelineRunner.lambda$run$1(SparkPipelineRunner.java:96)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      at java.lang.Thread.run(Thread.java:748)

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                ibzib Kyle Weaver
                Reporter:
                ibzib Kyle Weaver
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 2h 50m
                  2h 50m