[BEAM-10088] Examples Spark Runner tests failing [Java 11]

• Type: Sub-task
• Status: Open
• Priority: P3
• Resolution: Unresolved
• Component/s: runner-spark
• Assignee: Unassigned
• Reporter: Pawel Pasterz
• Created: 26/May/20 10:25
• Updated: 04/Jun/22 16:26

Description

Gradle *_:examples:java:sparkRunnerPreCommit _*fails during Java 11 Precommit job

Example stack trace:

> Task :examples:java:sparkRunnerPreCommit
    20/05/26 07:11:13 INFO org.apache.beam.runners.spark.SparkRunner$Evaluator: Entering directly-translatable composite transform: 'WordCount.CountWords/Count.PerElement/Combine.perKey(Count)'
    20/05/26 07:11:13 INFO org.apache.beam.runners.spark.SparkRunner$Evaluator: Entering directly-translatable composite transform: 'WriteCounts/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle'
    20/05/26 07:11:14 INFO org.apache.beam.runners.spark.metrics.MetricsAccumulator: Instantiated metrics accumulator: {
      "metrics": {
      }
    }
    20/05/26 07:11:14 INFO org.apache.beam.runners.spark.aggregators.AggregatorsAccumulator: Instantiated aggregators accumulator: 
    20/05/26 07:11:14 INFO org.apache.beam.runners.spark.SparkRunner$Evaluator: Evaluating Read(CompressedSource)
    20/05/26 07:11:14 INFO org.apache.beam.runners.spark.SparkRunner$Evaluator: Evaluating org.apache.beam.examples.WordCount$ExtractWordsFn@174ba402
    20/05/26 07:11:15 INFO org.apache.beam.runners.spark.SparkRunner$Evaluator: Evaluating org.apache.beam.sdk.transforms.MapElements$1@4c4d8e23
    20/05/26 07:11:15 INFO org.apache.beam.runners.spark.SparkRunner$Evaluator: Entering directly-translatable composite transform: 'WordCount.CountWords/Count.PerElement/Combine.perKey(Count)'
    20/05/26 07:11:15 INFO org.apache.beam.runners.spark.SparkRunner$Evaluator: Evaluating Combine.perKey(Count)
    20/05/26 07:11:19 INFO org.apache.beam.runners.spark.SparkRunner$Evaluator: Evaluating org.apache.beam.sdk.transforms.MapElements$1@6bfdc26a
    20/05/26 07:11:19 INFO org.apache.beam.runners.spark.SparkRunner$Evaluator: Evaluating Window.Assign
    20/05/26 07:11:19 INFO org.apache.beam.runners.spark.SparkRunner$Evaluator: Evaluating org.apache.beam.sdk.io.WriteFiles$WriteUnshardedTempFilesFn@57c4b142
    20/05/26 07:11:19 INFO org.apache.beam.runners.spark.SparkRunner$Evaluator: Evaluating GroupByKey
    20/05/26 07:11:19 WARN org.apache.beam.runners.spark.translation.GroupNonMergingWindowsFunctions: Either coder ShardedKeyCoder(VarIntCoder) or GlobalWindow$Coder is not consistent with equals. That might cause issues on some runners.
    20/05/26 07:11:19 INFO org.apache.beam.runners.spark.SparkRunner$Evaluator: Evaluating org.apache.beam.sdk.io.WriteFiles$WriteShardsIntoTempFilesFn@3122d00d
    20/05/26 07:11:19 INFO org.apache.beam.runners.spark.SparkRunner$Evaluator: Evaluating org.apache.beam.sdk.io.WriteFiles$WriteUnshardedBundlesToTempFiles$1@7639f2af
    20/05/26 07:11:19 INFO org.apache.beam.runners.spark.SparkRunner$Evaluator: Evaluating Flatten.PCollections
    20/05/26 07:11:19 INFO org.apache.beam.runners.spark.SparkRunner$Evaluator: Evaluating org.apache.beam.sdk.transforms.View$VoidKeyToMultimapMaterialization$VoidKeyToMultimapMaterializationDoFn@bcb712b
    20/05/26 07:11:19 INFO org.apache.beam.runners.spark.SparkRunner$Evaluator: Evaluating View.CreatePCollectionView

org.apache.beam.examples.WordCountIT > testE2EWordCount FAILED
    java.lang.IllegalArgumentException: Unsupported class file major version 55

1 test completed, 1 failed
Finished generating test XML results (0.002 secs) into: /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Java11_Phrase/src/examples/java/build/test-results/sparkRunnerPreCommit
Generating HTML test report...
Finished generating test html results (0.003 secs) into: /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Java11_Phrase/src/examples/java/build/reports/tests/sparkRunnerPreCommit