[BAHIR-139] Scala-maven-plugin does not respect Java compile level

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Spark-2.2.0
• Fix Version/s: Spark-2.2.1, Spark-2.3.0
• Component/s: Build
• Labels: build
• Environment: OpenJDK Runtime Environment (build 1.8.0_111-8u111-b14-2ubuntu0.16.04.2-b14) ; OpenJDK 64-Bit Server VM (build 25.111-b14, mixed mode)
• Assignee: Christian Kadner
• Reporter: Christian Kadner
• Created: 10/Oct/17 22:19
• Updated: 14/Oct/17 22:27
• Resolved: 14/Oct/17 22:27

Description

While working on PR #28 we found that the scala-maven-plugin does not respect the Java version (<java.version>1.8</java.version>) as configured by Java compile arguments -source and -target.

Bahir Scala-Maven-Plugin Configuration:

pom.xml

        <plugin>
          <groupId>net.alchim31.maven</groupId>
          <artifactId>scala-maven-plugin</artifactId>
          <version>3.2.2</version>
          ...
          <configuration>
            <scalaVersion>${scala.version}</scalaVersion>
            <recompileMode>incremental</recompileMode>
            <useZincServer>true</useZincServer>
            ...
            <javacArgs>
              <javacArg>-source</javacArg>
              <javacArg>${java.version}</javacArg>
              <javacArg>-target</javacArg>
              <javacArg>${java.version}</javacArg>
              <javacArg>-Xlint:all,-serial,-path</javacArg>
            </javacArgs>
          </configuration>

Compile Errors PR #28:

[bahir_pr28]$ mvn scala:compile -pl datasource-webhdfs

...
[ERROR] warning: [options] bootstrap class path not set in conjunction with -source 1.6

[ERROR] datasource-webhdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java:286: error: diamond operator is not supported in -source 1.6
[ERROR]       this.restCsrfMethodsToIgnore = new HashSet<>();
[ERROR]                                                  ^
[ERROR]   (use -source 7 or higher to enable diamond operator)

[ERROR] datasource-webhdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java:773: error: multi-catch statement is not supported in -source 1.6
[ERROR]           } catch (NoSuchMethodException | SecurityException 
[ERROR]                                          ^
[ERROR]   (use -source 7 or higher to enable multi-catch statement)

[ERROR] 2 errors
[ERROR] 1 warning
...

Maven DEBUG Output:

mvn -X compile -pl datasource-webhdfs

...
[INFO] --- scala-maven-plugin:3.2.2:compile (scala-compile-first) @ spark-datasource-webhdfs_2.11 ---
[DEBUG] Configuring mojo net.alchim31.maven:scala-maven-plugin:3.2.2:compile from plugin realm ClassRealm[plugin>net.alchim31.maven:scala-maven-plugin:3.2.2, parent: sun.misc.Launcher$AppClassLoader@74a14482]
[DEBUG] Configuring mojo 'net.alchim31.maven:scala-maven-plugin:3.2.2:compile' with basic configurator -->
[DEBUG]   (f) analysisCacheFile = /projects/bahir_pr28/datasource-webhdfs/target/analysis/compile
[DEBUG]   (f) args = [-unchecked, -deprecation, -feature]
[DEBUG]   (f) checkMultipleScalaVersions = true
[DEBUG]   (f) compileOrder = mixed
[DEBUG]   (f) displayCmd = false
[DEBUG]   (f) encoding = UTF-8
[DEBUG]   (f) failOnMultipleScalaVersions = false
[DEBUG]   (f) forceUseArgFile = false
[DEBUG]   (f) fork = true
[DEBUG]   (f) javacArgs = [-source, 1.8, -target, 1.8, -Xlint:all,-serial,-path]
[DEBUG]   (f) javacGenerateDebugSymbols = true
[DEBUG]   (f) jvmArgs = [-Xms1024m, -Xmx1024m, -XX:ReservedCodeCacheSize=512m]
[DEBUG]   (f) notifyCompilation = true
[DEBUG]   (f) outputDir = /projects/bahir_pr28/datasource-webhdfs/target/scala-2.11/classes
[DEBUG]   (f) pluginArtifacts = [net.alchim31.maven:scala-maven-plugin:maven-plugin:3.2.2:,...
[DEBUG]   (f) recompileMode = incremental
[DEBUG]   (f) scalaClassName = scala.tools.nsc.Main
[DEBUG]   (f) scalaOrganization = org.scala-lang
[DEBUG]   (f) scalaVersion = 2.11.8
[DEBUG]   (f) sendJavaToScalac = true
[DEBUG]   (f) session = org.apache.maven.execution.MavenSession@7a6ebe1e
[DEBUG]   (f) source = 1.6
[DEBUG]   (f) sourceDir = /projects/bahir_pr28/datasource-webhdfs/src/main/java/../scala
[DEBUG]   (f) target = 1.6
[DEBUG]   (f) useCanonicalPath = true
[DEBUG]   (f) useZincServer = true
[DEBUG]   (f) zincPort = 3030
[DEBUG] -- end configuration --
...

Notice:

[DEBUG]   (f) javacArgs = [-source, 1.8, -target, 1.8, -Xlint:all,-serial,-path]
[DEBUG]   (f) source = 1.6
[DEBUG]   (f) target = 1.6

Apparently the compile version defaults to Java 1.6 when the source and target elements are not explicitly set as part of the scala-maven-plugin <configuration>.

Proposed Fix:

Add the <source> and <target> elements to the scala-maven-plugin <configuration>:

            <source>${java.version}</source>
            <target>${java.version}</target>

pom.xml

        <plugin>
          <groupId>net.alchim31.maven</groupId>
          <artifactId>scala-maven-plugin</artifactId>
          <version>3.2.2</version>
          ...
          <configuration>
            <scalaVersion>${scala.version}</scalaVersion>
            <recompileMode>incremental</recompileMode>
            <useZincServer>true</useZincServer>
            ...
            <source>${java.version}</source>
            <target>${java.version}</target>
            <javacArgs>
              <javacArg>-source</javacArg>
              <javacArg>${java.version}</javacArg>
              <javacArg>-target</javacArg>
              <javacArg>${java.version}</javacArg>
              <javacArg>-Xlint:all,-serial,-path</javacArg>
            </javacArgs>
          </configuration>

Issue Links

• links to: GitHub Pull Request #51