Major Description
Execute
mvn test -pl mllib -Phadoop-3.2 -Phive-2.3 -Phadoop-cloud -Pmesos -Pyarn -Pkinesis-asl -Phive-thriftserver -Pspark-ganglia-lgpl -Pkubernetes -Phive -Pscala-2.13
6 java tests error:
[ERROR] Errors: [ERROR] JavaStreamingLogisticRegressionSuite.javaAPI:78 » TestFailed 20005 was not les... [ERROR] JavaStreamingKMeansSuite.javaAPI:78 » TestFailed 20040 was not less than 20000... [ERROR] JavaPrefixSpanSuite.runPrefixSpan:45 » NoClassDefFound scala/collection/parall... [ERROR] JavaPrefixSpanSuite.runPrefixSpanSaveLoad:67 » NoClassDefFound scala/collectio... [ERROR] JavaStreamingLinearRegressionSuite.javaAPI:77 » TestFailed 20014 was not less ... [ERROR] JavaStatisticsSuite.streamingTest:112 » TestFailed 20043 was not less than 200...
The error message is
Exception in thread "streaming-start" java.lang.NoClassDefFoundError: scala/collection/parallel/TaskSupport at org.apache.spark.streaming.util.FileBasedWriteAheadLog.<init>(FileBasedWriteAheadLog.scala:62) at org.apache.spark.streaming.util.WriteAheadLogUtils$.$anonfun$createLog$2(WriteAheadLogUtils.scala:125) at scala.Option.getOrElse(Option.scala:201) at org.apache.spark.streaming.util.WriteAheadLogUtils$.createLog(WriteAheadLogUtils.scala:123) at org.apache.spark.streaming.util.WriteAheadLogUtils$.createLogForDriver(WriteAheadLogUtils.scala:81) at org.apache.spark.streaming.scheduler.ReceivedBlockTracker.$anonfun$createWriteAheadLog$1(ReceivedBlockTracker.scala:265) at scala.Option.map(Option.scala:242) at org.apache.spark.streaming.scheduler.ReceivedBlockTracker.createWriteAheadLog(ReceivedBlockTracker.scala:263) at org.apache.spark.streaming.scheduler.ReceivedBlockTracker.<init>(ReceivedBlockTracker.scala:76) at org.apache.spark.streaming.scheduler.ReceiverTracker.<init>(ReceiverTracker.scala:114) at org.apache.spark.streaming.scheduler.JobScheduler.start(JobScheduler.scala:86) at org.apache.spark.streaming.StreamingContext.$anonfun$start$1(StreamingContext.scala:590) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18) at org.apache.spark.util.ThreadUtils$$anon$2.run(ThreadUtils.scala:218) Caused by: java.lang.ClassNotFoundException: scala.collection.parallel.TaskSupport at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) ... 14 more
Execute
mvn test -pl external/kafka-0-10 -Phadoop-3.2 -Phive-2.3 -Phadoop-cloud -Pmesos -Pyarn -Pkinesis-asl -Phive-thriftserver -Pspark-ganglia-lgpl -Pkubernetes -Phive -Pscala-2.13
1 java test error:
[ERROR] Errors: [ERROR] org.apache.spark.streaming.kafka010.JavaDirectKafkaStreamSuite.testKafkaStream [ERROR] Run 1: JavaDirectKafkaStreamSuite.testKafkaStream:170 expected:<[topic1-1, topic1-2, topic2-1, topic1-3, topic2-2, topic2-3]> but was:<[]> [ERROR] Run 2: JavaDirectKafkaStreamSuite.tearDown:57 » NoClassDefFound scala/collection/para...
The error message is:
[ERROR] Tests run: 2, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: 20.772 s <<< FAILURE! - in org.apache.spark.streaming.kafka010.JavaDirectKafkaStreamSuite [ERROR] org.apache.spark.streaming.kafka010.JavaDirectKafkaStreamSuite.testKafkaStream Time elapsed: 20.765 s <<< FAILURE! java.lang.AssertionError: expected:<[topic1-1, topic1-2, topic2-1, topic1-3, topic2-2, topic2-3]> but was:<[]> at org.apache.spark.streaming.kafka010.JavaDirectKafkaStreamSuite.testKafkaStream(JavaDirectKafkaStreamSuite.java:170) [ERROR] org.apache.spark.streaming.kafka010.JavaDirectKafkaStreamSuite.testKafkaStream Time elapsed: 20.766 s <<< ERROR! java.lang.NoClassDefFoundError: scala/collection/parallel/immutable/ParVector at org.apache.spark.streaming.kafka010.JavaDirectKafkaStreamSuite.tearDown(JavaDirectKafkaStreamSuite.java:57)
