Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-7820

Java8-tests suite compile error under SBT

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • 1.4.0
    • 1.4.1, 1.5.0
    • Build, DStreams
    • None

    Description

      Lots of compilation error is shown when java 8 test suite is enabled in SBT:

      JAVA_HOME=/usr/java/jdk1.8.0_45 ./sbt/sbt -Pyarn -Phadoop-2.4 -Dhadoop.version=2.6.0 -Pjava8-tests

      [error] /mnt/data/project/apache-spark/extras/java8-tests/src/test/java/org/apache/spark/streaming/Java8APISuite.java:43: error: cannot find symbol
      [error] public class Java8APISuite extends LocalJavaStreamingContext implements Serializable {
      [error]                                    ^
      [error]   symbol: class LocalJavaStreamingContext
      [error] /mnt/data/project/apache-spark/extras/java8-tests/src/test/java/org/apache/spark/streaming/Java8APISuite.java:55: error: cannot find symbol
      [error]     JavaDStream<String> stream = JavaTestUtils.attachTestInputStream(ssc, inputData, 1);
      [error]                                                                      ^
      [error]   symbol:   variable ssc
      [error]   location: class Java8APISuite
      [error] /mnt/data/project/apache-spark/extras/java8-tests/src/test/java/org/apache/spark/streaming/Java8APISuite.java:55: error: cannot find symbol
      [error]     JavaDStream<String> stream = JavaTestUtils.attachTestInputStream(ssc, inputData, 1);
      [error]                                  ^
      [error]   symbol:   variable JavaTestUtils
      [error]   location: class Java8APISuite
      [error] /mnt/data/project/apache-spark/extras/java8-tests/src/test/java/org/apache/spark/streaming/Java8APISuite.java:57: error: cannot find symbol
      [error]     JavaTestUtils.attachTestOutputStream(letterCount);
      [error]     ^
      [error]   symbol:   variable JavaTestUtils
      [error]   location: class Java8APISuite
      [error] /mnt/data/project/apache-spark/extras/java8-tests/src/test/java/org/apache/spark/streaming/Java8APISuite.java:58: error: cannot find symbol
      [error]     List<List<Integer>> result = JavaTestUtils.runStreams(ssc, 2, 2);
      [error]                                                           ^
      [error]   symbol:   variable ssc
      [error]   location: class Java8APISuite
      [error] /mnt/data/project/apache-spark/extras/java8-tests/src/test/java/org/apache/spark/streaming/Java8APISuite.java:58: error: cannot find symbol
      [error]     List<List<Integer>> result = JavaTestUtils.runStreams(ssc, 2, 2);
      [error]                                  ^
      [error]   symbol:   variable JavaTestUtils
      [error]   location: class Java8APISuite
      [error] /mnt/data/project/apache-spark/extras/java8-tests/src/test/java/org/apache/spark/streaming/Java8APISuite.java:73: error: cannot find symbol
      [error]     JavaDStream<String> stream = JavaTestUtils.attachTestInputStream(ssc, inputData, 1);
      [error]                                                                      ^
      [error]   symbol:   variable ssc
      [error]   location: class Java8APISuite
      

      The class JavaAPISuite relies on LocalJavaStreamingContext which exists in streaming test jar. It is OK for maven compile, since it will generate test jar, but will be failed in sbt test compile, sbt do not generate test jar by default.

      Attachments

        Issue Links

          Activity

            People

              jerryshao Saisai Shao
              jerryshao Saisai Shao
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: