Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-33772 Build and Run Spark on Java 17
  3. SPARK-37070

Pass all UTs in `mllib-local` and `mllib` with Java 17

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.3.0
    • 3.3.0
    • MLlib
    • None

    Description

      execute "mvn clean install -pl mllib-local"  with Java 17

      Run completed in 714 milliseconds.
      Total number of tests run: 96
      Suites: completed 9, aborted 0
      Tests: succeeded 92, failed 4, canceled 0, ignored 0, pending 0
      *** 4 TESTS FAILED ***
      

       

      execute "mvn clean install -pl mllib"  with Java 17

      Run completed in 10 minutes, 54 seconds.
      Total number of tests run: 1639
      Suites: completed 206, aborted 0
      Tests: succeeded 1635, failed 4, canceled 0, ignored 7, pending 0
      *** 4 TESTS FAILED ***
      

       

      All failed case has similar reasons:

      - rand *** FAILED ***
        org.mockito.exceptions.base.MockitoException: Mockito cannot mock this class: class java.util.Random.
      
      
      Mockito can only mock non-private & non-final classes.
      If you're not sure why you're getting this error, please report to the mailing list.
      
      
      
      
      Java               : 17
      JVM vendor name    : Azul Systems, Inc.
      JVM vendor version : 17+35-LTS
      JVM name           : OpenJDK 64-Bit Server VM
      JVM version        : 17+35-LTS
      JVM info           : mixed mode, sharing
      OS name            : Mac OS X
      OS version         : 11.4
      
      
      
      
      Underlying exception : java.lang.IllegalStateException: Cannot access annotation property public abstract boolean jdk.internal.util.random.RandomSupport$RandomGeneratorProperties.isStochastic()
        at org.scalatestplus.mockito.MockitoSugar.mock(MockitoSugar.scala:73)
        at org.scalatestplus.mockito.MockitoSugar.mock$(MockitoSugar.scala:72)
        at org.scalatestplus.mockito.MockitoSugar$.mock(MockitoSugar.scala:155)
        at org.apache.spark.ml.linalg.MatricesSuite.$anonfun$new$54(MatricesSuite.scala:807)
        at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
        at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
        at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
        at org.scalatest.Transformer.apply(Transformer.scala:22)
        at org.scalatest.Transformer.apply(Transformer.scala:20)
        at org.scalatest.funsuite.AnyFunSuiteLike$$anon$1.apply(AnyFunSuiteLike.scala:226)
        ...
        Cause: java.lang.IllegalStateException: Cannot access annotation property public abstract boolean jdk.internal.util.random.RandomSupport$RandomGeneratorProperties.isStochastic()
        at net.bytebuddy.description.annotation.AnnotationDescription$ForLoadedAnnotation.getValue(AnnotationDescription.java:663)
        at net.bytebuddy.implementation.attribute.AnnotationAppender$Default.handle(AnnotationAppender.java:298)
        at net.bytebuddy.implementation.attribute.AnnotationAppender$Default.doAppend(AnnotationAppender.java:361)
        at net.bytebuddy.implementation.attribute.AnnotationAppender$Default.append(AnnotationAppender.java:338)
        at net.bytebuddy.implementation.attribute.TypeAttributeAppender$ForInstrumentedType.apply(TypeAttributeAppender.java:93)
        at net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ForCreation.create(TypeWriter.java:5652)
        at net.bytebuddy.dynamic.scaffold.TypeWriter$Default.make(TypeWriter.java:2166)
        at net.bytebuddy.dynamic.scaffold.subclass.SubclassDynamicTypeBuilder.make(SubclassDynamicTypeBuilder.java:232)
        at net.bytebuddy.dynamic.scaffold.subclass.SubclassDynamicTypeBuilder.make(SubclassDynamicTypeBuilder.java:204)
        at net.bytebuddy.dynamic.DynamicType$Builder$AbstractBase.make(DynamicType.java:3595)
        ...
        Cause: java.lang.IllegalAccessException: class net.bytebuddy.description.annotation.AnnotationDescription$ForLoadedAnnotation cannot access interface jdk.internal.util.random.RandomSupport$RandomGeneratorProperties (in module java.base) because module java.base does not export jdk.internal.util.random to unnamed module @e55b13
        at java.base/jdk.internal.reflect.Reflection.newIllegalAccessException(Reflection.java:392)
        at java.base/java.lang.reflect.AccessibleObject.checkAccess(AccessibleObject.java:674)
        at java.base/java.lang.reflect.Method.invoke(Method.java:560)
        at net.bytebuddy.description.annotation.AnnotationDescription$ForLoadedAnnotation.getValue(AnnotationDescription.java:642)
        at net.bytebuddy.implementation.attribute.AnnotationAppender$Default.handle(AnnotationAppender.java:298)
        at net.bytebuddy.implementation.attribute.AnnotationAppender$Default.doAppend(AnnotationAppender.java:361)
        at net.bytebuddy.implementation.attribute.AnnotationAppender$Default.append(AnnotationAppender.java:338)
        at net.bytebuddy.implementation.attribute.TypeAttributeAppender$ForInstrumentedType.apply(TypeAttributeAppender.java:93)
        at net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ForCreation.create(TypeWriter.java:5652)
        at net.bytebuddy.dynamic.scaffold.TypeWriter$Default.make(TypeWriter.java:2166)
        ...
      

       

       

       

       

      Attachments

        Activity

          People

            LuciferYang Yang Jie
            LuciferYang Yang Jie
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: