Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-24417 Build and Run Spark on JDK11
  3. SPARK-28736

pyspark.mllib.clustering fails on JDK11

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0
    • Fix Version/s: 3.0.0
    • Component/s: MLlib, PySpark
    • Labels:
      None

      Description

      Build Spark and run PySpark UT with JDK11.

      $ build/sbt -Phadoop-3.2 test:package
      $ python/run-tests --testnames 'pyspark.mllib.clustering' --python-executables python
      ...
      File "/Users/dongjoon/APACHE/spark-master/python/pyspark/mllib/clustering.py", line 386, in __main__.GaussianMixtureModel
      Failed example:
          abs(softPredicted[0] - 1.0) < 0.001
      Expected:
          True
      Got:
          False
      **********************************************************************
      File "/Users/dongjoon/APACHE/spark-master/python/pyspark/mllib/clustering.py", line 388, in __main__.GaussianMixtureModel
      Failed example:
          abs(softPredicted[1] - 0.0) < 0.001
      Expected:
          True
      Got:
          False
      **********************************************************************
         2 of  31 in __main__.GaussianMixtureModel
      ***Test Failed*** 2 failures.
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                hyukjin.kwon Hyukjin Kwon
                Reporter:
                dongjoon Dongjoon Hyun
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: