Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-24417

Build and Run Spark on JDK11

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: 2.3.0
    • Fix Version/s: 3.0.0
    • Component/s: Build
    • Labels:
      None
    • Target Version/s:

      Description

      This is an umbrella JIRA for Apache Spark to support JDK11

      As JDK8 is reaching EOL, and JDK9 and 10 are already end of life, per community discussion, we will skip JDK9 and 10 to support JDK 11 directly.

        Attachments

          Issue Links

          1.
          Upgrade to Scala 2.11.12 Sub-task Resolved DB Tsai
          2.
          Upgrade SBT to 0.13.17 with Scala 2.10.7 Sub-task Resolved DB Tsai
          3.
          Upgrade ASM to 6.x to support JDK9+ Sub-task Resolved DB Tsai
          4.
          Accessing sun.misc.Cleaner in JDK11 Sub-task Resolved Sean R. Owen
          5.
          Add JDK11 in our Jenkins' build servers Sub-task Resolved Shane Knapp
          6.
          Upgrade ASM to 7.x to support JDK11 Sub-task Resolved DB Tsai
          7.
          install jdk11 on jenkins workers Sub-task Closed Shane Knapp
          8.
          Upgrade to Kafka 2.1.0 Sub-task Resolved Dongjoon Hyun
          9.
          Make Scala 2.12 as default Scala version in Spark 3.0 Sub-task Resolved DB Tsai
          10.
          Remove deprecated .newInstance(), primitive wrapper class constructor calls Sub-task Resolved Sean R. Owen
          11.
          Upgrading Hadoop to 2.7.4 to fix java.version problem Sub-task Resolved Takanobu Asanuma
          12.
          Upgrade Surefire plugin to 3.0.0-M2 Sub-task Resolved Dongjoon Hyun
          13.
          Fix core tests for Java 11 Sub-task Resolved Sean R. Owen
          14.
          Upgrade Mockito to 2.23.4 Sub-task Resolved Dongjoon Hyun
          15.
          Work around classloader changes in Java 9 for Hive isolation Sub-task Resolved Sean R. Owen
          16.
          Add maven profiles for running tests with JDK 11 Sub-task Closed Unassigned
          17.
          Upgrade to Kafka 2.1.1 Sub-task Resolved Dongjoon Hyun
          18.
          SizeEstimator can't make some JDK fields accessible in Java 9+ Sub-task Resolved Sean R. Owen
          19.
          Update JPMML to 1.4.8 for Java 9+ Sub-task Resolved Sean R. Owen
          20.
          Add JAXB reference impl to build for Java 9+ Sub-task Resolved Sean R. Owen
          21.
          Resolve Scala compiler failure for Java 9+ in REPL Sub-task Resolved Sean R. Owen
          22.
          Upgrade Maven to 3.6.1 Sub-task Resolved Dongjoon Hyun
          23.
          Upgrade ASM to 7.1 Sub-task Resolved Dongjoon Hyun
          24.
          No such method error (sun.nio.ch.DirectBuffer.cleaner()) when reading big table from JDBC (with one slow query) Sub-task Closed Unassigned
          25.
          No such method error (sun.nio.ch.DirectBuffer.cleaner()) Sub-task Closed Unassigned
          26.
          Upgrade Surefire plugin to 3.0.0-M3 Sub-task Resolved Dongjoon Hyun
          27.
          Change default class loader to ParallelGC Sub-task Closed Unassigned
          28.
          Remove `Illegal reflective access` warning for `java.nio.Bits.unaligned()` Sub-task Resolved Dongjoon Hyun
          29.
          Fix IncompatibleClassChangeError in `FromUnixTime` codegen on JDK9+ Sub-task Resolved Dongjoon Hyun
          30.
          Fix Flaky Test: `InputStreamsSuite.Modified files are correctly detected` in JDK9+ Sub-task Resolved Dongjoon Hyun
          31.
          on JDK11, IsolatedClientLoader must be able to load java.sql classes Sub-task Closed Unassigned
          32.
          Upgrade `xbean-asm7-shaded` to 4.14 Sub-task Resolved Dongjoon Hyun
          33.
          Upgrade janino to 3.0.13 Sub-task Resolved Yuming Wang
          34.
          Upgrade Mockito to 2.28.2 Sub-task Resolved Dongjoon Hyun
          35.
          MultilayerPerceptronClassifierTest.test_raw_and_probability_prediction fails on JDK11 Sub-task Resolved Hyukjin Kwon
          36.
          pyspark.mllib.clustering fails on JDK11 Sub-task Resolved Hyukjin Kwon
          37.
          Update jersey to 2.27+ (2.29) Sub-task Resolved Sean R. Owen
          38.
          test_mllib_classification fails on JDK11 Sub-task Resolved Hyukjin Kwon
          39.
          Fix checkJavaVersion to accept JDK8+ Sub-task Resolved Dongjoon Hyun
          40.
          add java11 support for spark pull request builds Sub-task Resolved Shane Knapp
          41.
          Dependency generation for JDK8/JDK11 Sub-task Resolved Dongjoon Hyun
          42.
          Investigate/fix JAXB failure running Pyspark tests on JDK 11 Sub-task Resolved Dongjoon Hyun
          43.
          Maven install fails on JDK11 Sub-task Resolved Dongjoon Hyun
          44.
          Upgrade scala-maven-plugin to 4.2.0 Sub-task Resolved Hyukjin Kwon
          45.
          Add more profiles for JDK8/11 build test Sub-task Resolved Dongjoon Hyun
          46.
          Upgrade Scala to 2.12.10 Sub-task Resolved Yuming Wang
          47.
          Update dropwizard metrics to 4.1.1 for JDK 9+ support Sub-task Resolved Sean R. Owen

            Activity

              People

              • Assignee:
                dbtsai DB Tsai
                Reporter:
                dbtsai DB Tsai
              • Votes:
                28 Vote for this issue
                Watchers:
                71 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: